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ABSTRACT 

This report presents a summary of activities, 
findings, and recommendations from Phase I of a project to improve 
the effectiveness of academic health science libraries and their 
provision of information services. The objectives for Phase I of the 
project are given: (l) to identify academic health science library 
requirements in the areas of planning, performance measures, and 
management information systems; (2) to analyze the Annual Statistics 
database in light of these requirements; (3) to develop draft 
performance measures, service roles, and planning models; (4) to 
conduct a workshop to inform the membership of the Association of 
Academic Health Science Library Directors (AAHSLD) on the status of 
the project, as well as to inform them on the components of the 
planning process; and (5) to task the remaining phases of the 
project. The main body of the report provides a brief introduction to 
the project, an overview of the method, key findings and conclusions, 
and a proposed tasking and budget for remaining project phases. 
Twenty key recommendations cover strategic planning, vision 
statements and performance measurement, data management, and improved 
education for management and evaluation. The seven appendices, which 
make up the major part of the final report, contain detailed reports 
on data collection activities, the findings, and recommendations. 
(KRN) 
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A NEW STRATEGIC DIRECTION FOR THE AAHSLD ANNUAL STATISTICS 



PLANNING, SERVICE ROLES, PERFORMANCE MEASURES, AND 
MANAGEMENT INFORMATION SYSTEMS FOR ACADEMIC HEALTH SCIENCE LIBRARIES: 

FINAL REPORT FOR PHASE I 



This report presents a summary of activities, findings, and recommendations from Phase I of the 
project "A New Strategic Direction for the AAHSLD Annual Statistics: Planning, Service Roles, 
Performance Measures, and Management Information Systems for Academic Health Science Libraries." 
Phase I of the project began in January, 1991 and was completed in December, 1991. The overall purpose 
of the project is to improve the effectiveness of academic health science libraries and their provision of 
information services. More specifically, the project has the objectives of: 

• Producing a practical manual that integrates planning, vision /service role setting, 
performance measurement, and managing data 

• Redesigning the AAHSLD Annual Statistics database to collect and report a range of 
performance measurements, and to better manage library statistics 

• Educating AAHSLD member staff about the uses of strategic planning, vision/service 
role setting, performance measures in academic health science library decision making 
and in the provision of information services. 

The AAHSLD Annual Statistics Editorial Board served as an advisory committee to the study team 
during Phase I. 

The project builds upon the work reported in Challenge to Action: Planning and Evaluation ' 
Guidelines for Academic Health Sciences Libraries (Chicago: Association of Academic Health Science 
Library Directors and the Medical Library Association, 1987). The guidelines identified in that report 
recognized the need for improved strategic planning and performance assessment for academic health 
science libraries. The Study Team and the AAHSLD Annual Statistics Editorial Board considered the 
work in this report as a basis for designing the project described herewithin. 

The project, as originally proposed, consisted of four phases. As the study team worked on Phase I, 
it became apparent that some of the tasks originally scheduled for Phase II could also be completed. 
Thus, this report on Phase I incorporate some of the activities originally scheduled for Phase II. The 
revised objectives, then, for Phase I were: 

• Identify academic health science library requirements in the areas of planning, performance 
measures, and management information systems 

• Analyze the Annual Statistics database in light of these requirements 



• Develop draft performance measures, service roles, and planning models 
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• Conduct a workshop to inform the AAHSLD membership on the status of the project as well as 
to inform them on the components of the planning process 

• Task the remaining phases of the project. 

An underlying objective throughout Phase I was to provide a basis for obtaining additional external 
funding to support the completion of the project after Phase I had been completed. 

The final report for Phase I includes a number of component parts. The main body of the final 
report provides a brief introduction to the project, an overview of the method, key findings and 
conclusions, and a proposed tasking and budget for remaining project phases. The literature review and 
assessment. Planning. Per formance Measures, and Data for Decision Making: Key Sources an d Issues for 
Academic Health Science Libraries (Information Management Consultants Services, Inc.- June 15, 1991), 
is a 200+ page report and has been delivered separately to AAHSLD. The detailed description of data 
collection activities, findings, and recommendations are contained in a number of appendices. These 
appendices originally were distributed to the Statistics Editorial Board for review and comment; their 
suggestions have been incorporated in the final versions attached to this report. 



THE MANAGERIAL IMPERATIVE 

One of the most important challenges facing academic health science library (AHSL) managers is 
to improve overall effectiveness in and overall productivity of their libraries. One of the best possible 
strategies for meeting this challenge lies in the development of an approach that integrates strategic 
planning, develops vision statements and service roles, assesses the library with performance measures, 
and manages/relates data about each of these activities in a management information system (MIS). 
Such a comprehensive and integrated approach to library management is essential if AHSLs are to 
provide the leadership in accessing, managing, and disseminating health science, information to their 
clients effectively in the future. 

The library management process includes, minimally, identifying clientele information needs, 
articulating a vision of innovative library services, setting objectives to accomplish a range of library 
activities, and demonstrating the degree of effectiveness and efficiency with which the objectives 
have been accomplished and the impact of the accomplished objectives on the clientele served. This 
notion of management will become increasingly important as demands for new services and activities 
increases, resources become scarce, governing bodies demand better and more accountability for library 
services, and basic cosls for library resources and technology increase. 

Findings from Phase I of this project suggest that there is wide interest among AHSL staff to 
improve the management process in their libraries. Indeed, many of the study participants identified 
the key factors (also confirmed by the study itself) that have limited the success of AHSL management 
efforts: 

• Limited planning efforts: for a host of reasons, many AHSLs do not have a formalized planning 
process or a concise and coherent plan that articulates the library's vision of itself and how it 
will deal with the future; there is a wide range of views as to what constitutes a "strategic 
plan" and what should be in such a plan 
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• Unclear priorities: the limited planning efforts have left many libraries trying to do all things 
for all people, all the time; without priorities, it is unclear which library activities and 
services are most important 

• Limited evaluation efforts: many libraries simply do not know how well they meet user 
information needs, the level of productivity of staff, costs for basic types of services, the impact 
of the library on larger institutional goals, or the degree to which they are successfully 
accomplishing objectives; this situation is exacerbated by a lack of standardized and agreed 
upon performance measures for AHSL activities and services 

• Lack of integrated management data : libraries often have inadequate management data, the 
data available typically stress inputs to the library rather than outputs, outcomes, or overall 
productivity, and the data are not easily related to each other (e.g., types of client groups that 
use specific services at what specific costs); there is a need for AHSLs to develop their own MIS 
as well as be able to better rely on the AAHSLD Annual Statistics data base 

• Inadequate training and education : library managers require additional education and training 
in the areas of strategic planning, performance measurement, design and implementation of 
MIS, uses of data for decision making, and integrating the strategic planning process into the 
larger institutional planning process. 

While other factors also contribute to managerial effectiveness, these five suggest the need for a 
comprehensive and integrated management approach for AHSLs, an approach not currently available. 

Findings from Phase I of this study emphasize the nead to go beyond the collection and production 
of AAHSLD statistics, to go beyond producing measures out of context of management needs and client 
services, and to place statistics development in a larger context of AHSL and institutional management. 
Strategic planning, vision statements, performance assessment, management information systems, and 
AAHSLD statistics gathering and use must be better coordinated and integrated if AHSLs are to better 
meet future challenges of health science information management. 



METHODOLOGY 

The first phase of the project was designed to meet a number of particular objectives. These 
objectives are described above. The accomplishment of the objectives required the study team to 
undertake a variety of data collection activities. The data collected were then analyzed to identify 
key themes and issues for planning activities in academic health science libraries. Following these 
analytic tasks, draft performance measures and vision/service roles were developed and refined and a 
workshop for the membership presented. 



Data Collection Activities 

A variety of data collection activities was undertaken in order to provide the data necessary to 
assure accomplishment of the objectives. Particular data collection activities undertaken during Phase 
I of the project were: 
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• A review of the literature on strategic planning, performance measurement, and uses of data for 
decision making in health science libraries (report issued separately) 

• A review of findings from A Study of the Use of AAHSLD Annual Statistics in Mem ber 
Libraries, conducted by Hert and McClure (IMCS, Inc., 1990) 

• An analysis of planning documents from AAHSLD member libraries (Appendix B) 

« The conducting of four focus groups, three with academic health science librarians and one with 
health science center administrators (Appendices C and E) 

• Assessment of preliminary findings by AAHSLD November 9, 1991, workshop attendees 
(Appendix H). 

Details on the method, data collection strategies, and findings/recommendations in contained in the 
summary reports included in the Appendixes. 

The identification of academic health science library requirements in the areas of planning, 
performance measures, and management information systems was the critical first step in determining 
possible performance measures, planning models, and service roles that would be useful for academic 
health science libraries. Many of the data collection activities of Phase I were designed to illuminate 
these requirements. 

The first data collection activity undertaken was a review of the literature on planning in health 
science libraries. This review indicated several important factors which influenced planning activities 
in medical libraries and which might have some relevance for academic health science library 
planning activities specifically. 

A literature review provides valuable information about the general context of a given problem. It 
does not, however, provide evidence of what is actually occurring in particular libraries, nor does it 
capture what those working in particular situations feel to be the relevant issues for their planning 
activities. In order to flesh out the preliminary information provided by the literature review, the 
study team reviewed a previous study of the AAHSLD Annual Statistics, analyzed existing planning 
documents, and conducted several focus groups. 

This project reviewed a previous study of the AAHSLD Annual Statistics which was performed by 
Carol A. Hert and Charles R. McClure for the Statistics Committee of the AAHSLD in May 1990 
(Final report entitled: A Study of the Use of the AAHSLD Annual Statistics in Member Libraries) 
[available through ERIC Clearinghouse as ED 325 141]. This survey provided data on how AAHSLD 
members were currently using quantitative data and the Annual Statistics, possible changes which 
could be made to the volume so that its usefulness to members could be improved, and provided data on 
the current and potential use of performance measures for decision making. 

Planning documents were submitted by 33 member libraries of the AAHSLD in response to a letter 
from the AAHSLD Executive Board. The libraries submitting plans represented a range from small to 
large in terms of size of institution, number of primary clients, and total number of volumes. These plans 
were examined specifically to determine how well they met generally agreed-upon criteria for what 
constitutes an effective strategic plan. 

5 
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The study team also conducted four focus groups. Focus groups consist of MO people possessing 
specific characteristics (e.g. all participants perform administrative functions in an academic health 
science library) who provide data of a qualitative nature in a focussed discussion. Focus groups have 
several valuable advantages as a data collection technique. Unlike interviews with predetermined 
questions, focus group discussions rely on the participants to define the topics and issues which are 
important to them. This process is critical when it is not clear what all the relevant issues might be. 
Additionally, during a focus group, participants interact with each other and that interaction leads to 
a greater involvement of all participants and a concentration on the participants' points of view. 
Having an accurate representation of real points of view is critical to the development of a planning 
manual which will be useful to the AAHSLD membership. 

Three focus groups were held with academic health science librarians from a variety of 
institutions. One session was held with health science center administrators. Topics discussed at these 
focus group sessions covered a range of areas and were intended to explore issues related to planning, 
performance measurement, data management, and perceptions of the academic health science library in 
the larger institutional setting. Summary reports from these sessions are found in Appendices C and E. 

An innovative data collection strategy included the review and assessment of preliminary project 
findings at the November 9, 1991 workshop. This workshop was attended by approximately 110 
academic health science librarians. In addition to presenting the findings from Phase I of the project, 
participants meet in small groups to review and assess these findings. This assessment included a 
review of draft vision statements, criteria describing institutional settings, and possible AHSL 
.performance measures. Due to workshop participant interest, the summary report was made available 
to AAHSLD, to provide an edited version of the results for participants. The complete report of the 
results from this data collection activity appear in Appendix H. 



Analysis of the AAHSLD Database and Performance Measure Development 

The study team developed possible performance measures for academic health science libraries. 
The Annual Statistics database was analyzed to determine what performance measures might be 
produced from already existing data. Additionally, an analysis of data from the focus groups revealed 
that measures which demonstrated outcomes of library services in relation to fulfillment of 
institutional missions are important to improvement of library planning activities. These activities 
have resulted in the articulation of an approach to performance measure development which is 
discussed in more detail in Appendix D. 

Development of Vision Statements and Planning Models 

Draft service roles and vision statements were also developed by the study team. The development 
of these vision statements were informed by a review of already existing service roles developed by the 
Texas Medical Center Library (with the assistance of Information Management Consultant Services, 
Inc.) as well as by data collected in the focus groups. These draft vision statements were presented to 
the AAHSLD membership during the November 9, 1991 workshop for their feedback. The draft vision 
statements are included as Appendix F. 
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The study team also worked to develop models of a planning process for AAHSLD member libraries. 
This effort involved a consideration of the interrelationships between vision statement development, 
performance measures, institutional characteristics affecting strategic planning, and the information 
systems used to collect, store, and analyze associated data. An initial explication of one possible model 
was developed and presented at the workshop. 



Workshop Presentation 

The final major objective of Phase I was to present a workshop to inform the AAHSLD membership 
of the status of the project and to focus attention on the planning process for academic health science 
libraries. Workshop content and structure were developed based on findings from the data collection 
activities. Material for the workshop was mailed to participants prior to the workshop. An overview 
of the material developed for the workshop and the various group exercises used at the workshop are 
included in Appendices F and G. 



KEY RECOMMENDATIONS AND FINDINGS 

The various data collection activities and the subsequent analysis which were the core of Phase I of 
the projeV . resulted in a number of findings and recommendations for future action in the remaining 
phases Ok <he project. These findings and recommendations can be grouped into four main categories 
which represent the four main objectives of the project. These categories are strategic planning, service 
role and performance measure development, data management, and education. 



1. Strategic Planning 

A. Planning efforts at the library and institutional level are not generally well developed. 

The study team generally found that strategic planning activity both within the library and 
within the institution was scattered and still fairly undeveloped. There still appears to be a great 
reliance on anecdotal reports (which may be backed up with quantitative data). Planning terminology 
and concepts frequently mean different things at different libraries and institutions. Institutional 
administrators appear to not know what to ask for from libraries and are waiting for instruction. 

B. Health science libraries must think strategically. Planning must focus on key issues of concern to the 
library and the institution. 

Librarians must be able to solve problems for people in their environment. This means that they 
must be in the right place at the right time with the right information. This entails monitoring their 
environment in order to know what the problem is, the right place is, what timing will work, and what 
information will solve the problem. Planning must assist librarians in their monitoring efforts, halping 
them to understand their environments and think strategically about what to do next. 
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Planning must be focussed on key issues of concern to the library and the institution. For many 
libraries, this may mean a focus on specific issues and outcomes (such as space utilization) rather than 
long-term strategic positioning. A focus on specific issues and outcomes may be very important in 
academic health science libraries where the culture emphasizes specific outcomes in specific time 
frames. 



C For medical libraries to survive requires that they play a more visible role In the life of the medical 
center. Assistance is needed in developing strategies and techniques for gaining and retaining 
participation in institutional planning and management efforts. 

The library and its staff need to become full partners in institutional management activities if they 
wish to remain a viable part of the organization. The value of the library to various institutional 
activities needs to be demonstrated and library staff need to become active participants in planning and 
management at the institutional level. Library managers need to be, and see themselves as managers 
within the larger institution. 

The library must also change the medical center administrators' perception of "overhead," and the 
sense that the library, somehow, is "overhead." Participating fully within the institution brings with 
it the responsibility to act in a different manner, providing clear indication of the library's value to the 
organization. In order to accomplish this goal, quantitative measures of the library's value to the 
institution and the larger medical environment need to be developed. 



D. Links between library service and institutional goals need to be developed. 

One of the most effective ways that libraries can demonstrate their value to the larger 
organization is through the creation of evaluative measures which link library services to specific 
institutional goals. . Throughout the data collection activities in Phase I of the project, a number of 
possible institutional missions and goals to which library services might be linked were identified. But 
such linkages need to be more fully explicated in the form of vision statements in order to facilitate 
effective library management. 

E. Library staff must overcome a number of barriers to become integrated in institutional planning 
activities. 

Library staff face several problems in becoming integrated into planning efforts including 
institutional barriers (e.g., committees limited to faculty participants only), lack of planning and data 
management skills and the lack of appropriate measurement data. Librarians frequently feel isolated 
from existing planning efforts in the institution for these reasons. Mechanisms need to be developed in 
the library's strategic planning process that integrate and link them better to institutional goals and 
activities. 
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F. The unique institutional setting of a library plays a critical role in determining the nature of 
planning activities. 

In the focus groups, it became clear to the study team that an understanding of the institutional 
environment of an academic health science library is critical to understanding how strategic planning 
might be effective. Institutional features which may affect planning include: i 

• The reporting lines of the library within the institution 

• The institutional culture 

• Whether the institution is public or private 

• Whether decision making is centralized or decentralized 

• The professional background of top decision makers 

• The level of available funding 

• The source of funding (e.g., state, research, clinical) 

• The degree of library independence from its administration 

• The clientele served 

• The strategic issues and priorities of the institution 

• The age of the institution 

• The institution's commitment to planning 

• Whether the institution has single or multiple programs 

• The relationship between the hospital and the medical school 

• Whether the institution had 2 or 4 year programs 

• Whether the institution has an urban or rural setting 

• Whether the institution is research intensive or teaching oriented 

• Whether the institution is centralized on one campus or distributed geographically. 

The range of such factors suggest that since every organization is different to some extent, a planning 
manual must consider how such institutional factors affect successful planning and performance 
measurement, and offer suggestions for modifying planning and measurement in light of these 
conditions. 

The study team believes that it may be possible to determine which of these factors are most 
important for planning activities and create a typology of institutional settings utilizing several of the 
factors which could be incorporated into the planning manual. 

G. There is a crisis in medical school administration which adversely affects the library's ability to 
plan successfully. 

High turnover in the medril school administration has affected the library's ability to interpret 
the organizational climate and then plan based on that interpretation. Such a problem may argue for 
the development of planning and evaluation tools and techniques which function in the short run rather 
than the long run. Medical school administrations have not determined, for themselves, how to 
measure the school's performance. This makes it exceedingly difficult for the library to develop 
measures of effectiveness which are tied to institutional goals. There may be a lack of adequate 
education and training in managerial activities on the part of institutional managers. The library 
might serve a valuable function in the education of institutional managers both through provision of 
information resources as well as through the demonstration of effective and efficient planning. 

9 
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2. Vision Statements and Performance Measurement 

A. The library should articulate a strategic vision to its institution but that vision should be developed 
in partnership with other institutional stakeholders. 

Librarians and administrators stressed the need for library services to be guided by a clear vision or 
set of service roles for the library. That vision should be "exciting," "challenging," and able to better 
link the library into institutional goals. Administrators were concerned that this vision needed to be 
developed with input from all stakeholders. Such a process would provide a valuable foothold for 
library involvement in institutional planning activities. 

B. New service roles should be Identified for academic health science libraries. 

There is a need for new visions of library activities and services. Many of these visions may need to 
better describe and exploit new technologies and allow remote access/use of the library. There may be a 
variety of ways for the library to link to institutional missions which could be encapsulated into 
service roles which might serve as focussed visions incorporating a number of services and activities. 
The draft set of vision statements which were presented at the workshop (see Appendix F) are a first 
effort here, but need to be more "visionary." 



C. AAHSLD libraries require better measures for performance for evaluating library services- 
espedally measures focussing on user needs and library impacts on the institution. 

There is general agreement that the concern for evaluating libraries must translate into research 
and development related to performance measurement. The research should consider library 
performance in terms of meeting user information needs, library outputs, and library outcomes. Special 
attention should be given to developing measures that indicate the contributions and impact of the 
library on institutional goals and services. 



D. Performance measures should be linked to particular library service roles or visions. 

While it is evident that there are a range of performance measures for AHSLs, in general, there is 
also a need for performance measures especially appropriate for the particular needs of an individual 
library. One role of the AAHSLD data base is to assist in collecting and analyzing data to produce 
these "generic" performance measures that may be of use to all or most the AAHSLD libraries. Other 
performance measures, however, should be developed at a local level and will require the library to 
determine which data it needs, how it should best be collected, what type of a MIS should be 
developed, and how the MIS will best support library decision making and planning. 



10 



11 



IMCS, Inc. 



AAHSLD Phase I Final Report 



Decembers, 1991 



E. Measures should support a variety of planning activities and should provide indicators of library 
"quality" as well as comparisons among AAHSLD member libraries. 

Traditionally, performance measures have been used by libraries to compare themselves to other 
similar libraries. While this purpose for performance measures is an important one, it is important not 
to lose sight of other uses for performance measures. Both the literature review and focus group sessions 
with institutional administrators suggest the need for performance measures that demonstrate the 
impact and contribution of the library to institutional goals. Attention to developing a strategic 
planning process and performance measures that integrate and link the library to the larger 
institutional context is essential. 



3. Data Management 

A* Data must be collected for a purpose; data collection must be problem-centered. 

Data collection efforts need to focus on particular problems to be resolved rather than on measuring 
processed. In considering what data to collect, librarians should first consider what their top problems 
are and then consider what data will help them in thinking about and solving these problems. This 
suggests that a planning manual should include information on how to prioritize problems and 
determine what types of data might be useful in resolving them. 

B. Data to be collected by AAHSLD member libraries should be stable, and accommodate a wide range 
of needs. However, data collection efforts also must keep up with changes in technology, the health 
sciences, and within the Institution. 

Both the current project and previous research by Hert and McClure indicated the need for measures 
and data collection which remains stable over time. Time series analysis is only possible when 
measures have remained constant over several years. This concern must be balanced with the need to 
provide data and corresponding performance measures which capture information about new 
initiatives, changing problems, new technologies, and environmental changes within the library and 
the institution. 

C. The level of effort to collect data must be minimal, the directions for collection standardized and 
easy to use. 

Given competing demands on staff time and resources, data collection should occur with minimal 
effort and procedures for collection need to be well documented. These needs are particularly important 
for data collected for the AAHSLD since they are used for comparative purposes. This means that 
AAHSLD libraries must carefully determine which data will be collected, insure that the data are 
collected and analyzed in a standardized manner, and that the reporting of the data is straight- 
forward. 
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D. The AAHSLD database must be seen as one data source; its partner is a MIS at the local academic 
health science library. 

The range of data collection activities, statistics, and performance measures needed by individual 
AAHSLD libraries cannot be met only by the AAHSLD data base. The AAHSLD data base should 
provide nationally relevant information, provide a basis for valid and meaningful comparisons among 
AAHSLD libraries, and provide data on selected, key performance measures. Individual AAHSLD 
libraries will need to take responsibility for the development and operation of a MIS at their library 
that (1) meets local planning and performance measurement needs, and (2) compliments the data in the 
AAHSLD data base given the library's particular data collection needs. 

E. The existing AAHSLD database should be made more "performance measurement oriented" and 
collect only that data that i> most important for identifying national trends and for making 
meaningful caparisons among AAHSLD libraries. 

An implication of this recommendation is that probably less data should be collected for inclusion 
in the AAHSLD database, but those data should be carefully determined in light of (1) appropriate 
data to be collected locally versus nationally, (2) the data needed to produce meaningful performance 
measures, and (3) the data that relate best to general, or representative types of service roles. Given 
the resources needed, and level of support available for developing and maintaining this database, 
some costs and commitments might best be shifted to individual AAHSLD libraries from the AAHSLD 
national effort. 

F. Library data management techniques appear to be evolving and being recognized v - important 
management activity. 

From the various data collection activities as well a:> the workshop, it is clear that there is 
increased recognition of the importance of data management. The demonstration of the MIS at the 
workshop brought home the notion that improved data management techniques were, in fact possible 
and doable. Still to be accomplished, however, is a clear and straightforward set of guidelines for how 
data management can be best accomplished. 

4 Education 

A. Improved education and training in the area of planning and evaluation for academic health science 
library services is essential. 

Underlying much of the study team's work was the concern that both library and institutional 
managers are unprepared for the managerial responsibilities they have to fulfill. Thus, the 
development of a manual for academic health science libraries should be seen in the context of 
educating academic health science librarians as well as simply producing a manual. The educational 
imperative includes an attitudinal component. Some academic health science library managers have 
"doubts" as to the usefulness and/or appropriateness of strategic planning , performance measurement, 
and data management. 
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B. AAHSLD, MIA, individual library directors, and others must provide a leadership stance in the 
development of management education and skills. 

Phase I found a wide range of views and knowledge regarding strategic planning, performance 
measurement, MIS, and uses of data for library decision making among AAHSLD managers. Moreover, 
it is unclear how specific programming efforts among these key stakeholders are being coordinated to 
better prepare AHSL managers to better manage their libraries for the future. This coordinating effort, 
as well as the leadership and vision to organize and implement such a program could be fulfilled by 
AAHSLD. 



TASKING 

This section offers an overview of tasking and project management necessary to complete the 
development of the manual as outlined in the original proposal (December 20, 1990) and in light of 
findings from Phase I. This tasking is an overview and it is likely that the first months of the project 
would be spent on detailed tasking and specification of the tasks and objectives outlined in this section. 
Based on our experience with Phase I, we would propose that the project be completed in one final 
phase of two years. The nature of the work to be completed, i.e., different components closely linked to 
each other, argues against breaking the tasks into two or three additional phases as originally 
planned. 

In addition to the detailed tasking that would occur in the first few months, the study team should 
also review and analyze the findings and documents included in the appendices of this report. 
Summary assessments from the workshop findings, incorporated with findings from the other findings 
from Phase I may help to refine and detail the tasks to be completed in the remainder of the project. 

Projects 

Overall, the product from the study will be an integrated management approach to planning and 
measurement for AAHSLD libraries, consisting of: 

• A manual describing: 

- an integrated strategic planning and performance measurement process 

- a set of parent institutional types to be used in planning, evaluation, and communications 
decisions; 

- a process to develop vision statements with a set of example vision statements to be used in 
developing a library's own vision 

- a process and set of guidelines by which a library would design its own management 
information system (MIS) 

• Guidelines for reorganizing the AAHSLD database in light of the performance measures 
recommended for inclusion in the manual 

• Implementation of an educational program during the project to inform and educate AAHSLD 
librarians about the management system. 
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The development of these components will occur in parallel, and many specific activities will be 
designed to further more than one component At the end of the project these three components will 
come together into an integrated system. However, the process of the project is most clearly described 
by addressing the following key components individually. 



Strategic Planning 

GOAL- to produce a manual describing an integrated planning and measurement process. 

The planning process will basically follow the steps outlined at the November 9, 1991 workshop and 
the steps described in the evaluation of planning documents (see Appendices). 

Two significant findings of Phase I were: 

• The identification of the critical role of parent institutional type in library planning and 
evaluation, particularly in how the library communicates with its parent organization; 

• The importance of vision statements and the need for AAHSLD libraries to have a process for 
developing their own vision statements. 

The Planning task will extend upon these results and integrate them into the planning approach being 
described. 

Objectives: 

• Develop a typology of institutional settings, including the relevant dimensions on which 
institutions are to be distinguished, the implications of institutional type for library planning- 
and communication with its parent institution, and a process by which a library could assess its 
own setting 

• Develop a set of example vision statements and/or components of a vision statement, to be used 
by a library in developing its own vision 

• Describe a process by which a library would construct its own vision statement 

• Develop and describe a planning process appropriate to the AAHSLD libraries, incorporating 
institutional typology and vision statement development 

• Test the planning process in at least three libraries of different sizes and different parent 
institutional types (the purpose of the test is to give immediate feedback on the developing 
planning process, with special attention to' the most innovative ,and therefore untried, parts of 
this project the institutional setting typology and the vision statements). 

• Produce a draft manual that describes the planning process, including the assessment of 
institutional typology, development of a vision statement, and measurement 

• Complete the manual. 
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Performance measures/ AAHSLD database 

GOAL: to develop specifications for a revised AAHSLD statistical database and report. 
Objectives: 

• Identify the (1) enduring and (2) current problem-based data needs of AAHSLD libraries from 
the AAHSLD database using 

- the sample vision statements from task I 

- the experiences of the test libraries in task I 

- the Hert/McClure assessment of the database and the AAHSLD Statistical Report 

- other information collected as a part of Phase I of the project. 

• Assess the reliability of the current data in the AAHSLD database 

• Test, refine, and validate the performance measures 

• Develop a set of specifications for the content of the AAHSLD database - including data 
elements and general data collection methods. 

Management Information Systems 

GOAL: develop guidelines for individual AHSL libraries to develop their own management 
information systems (MIS). 

Objectives: 

• Describe a process by which each library can assess its own needs from an MIS system - a 
process that may be integrated with the planning process developed in Task 1 

• Develop a description of the components of an AAHSLD MIS system — that is, the kinds of 
data that an AAHSLD library should consider as candidates for an MIS system 

• Integrate the MIS system development process with the planning and performance measurement 
process described above and include the documentation of the MIS development process in the 
planning manual 
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Education 

GOAL: develop an educational program to inform and educate AAHSLD librarians about the project, to 
run parallel with the development of this project, and to culminate in training in the use of the manual. 



Objectives: 

• Provide ongoing dissemination of the project goals and progress in the AAHSLD Newsletter 

• Develop informative programs about the project and its progress at MLA 

• Develop educational workshops and programs about the project at the AAHSLD/ AAMC 
annual meetings 

• Train the trainers upon publication of the manual. 

• Offer a workshop for AAHSLD library staff at the AAHSLD/ AAMC meeting following the 
publication of the manual. 



Overview 

Tasking falls into these four categories as outlined above. Figure 1 offers a more detailed view of 
the key tasks with a corresponding time schedule that might be proposed to complete the project. 
Again, we note that additional detailed tasking is needed and would be completed during the first few 
months of the project in light of a careful review of Phase I findings and recommendations-as given in 
the appendices. 

The tasking described in this section is based on the proposed budget outlined in the following 
section. Significant changes in the budget would require a reassessment of the tasking and level of 
effort dedicated to those tasks. 



PROPOSED BUDGET 

The budget proposed in figure 2 assumes a two year time period to complete the project. Given the 
range of activities to be accomplished (see tasking above), we believe the level of effort in terms of 
personnel time commitment would be needed. The amount proposed does not include overhead or fringe 
on personnel. Depending on the funding source(s), the contracting institution, and the actual individuals 
doing the work, additional amounts may be required to cover these two expenses. 
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FIGURE 1: OVERVIEW OF TASKS 
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FIGURE * OVERVIEW OF TASKS 
(Continued) 
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in. mis 
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FIGURE 1: OVERVIEW OF TASKS 
(Continued) 
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IMPORTANCE OF COMPLETING THE PROJECT 

AAHSLD has a unique opportunity to take a leadership stance for improving health sciences 
library effectiveness and provision of information services. An integrated approach to planning, service 
role selection, performance measures, and MIS has the potential to be the single most important 
management strategy for the 1990s. Indeed, it is a strategy that will greatly assist health science 
libraries to cope with uncertain futures, allocate resources more effectively, provide a rational basis for 
determining which types of information services are to be provided to which clientele groups, and 
demonstrate accountability and needs to library governing bodies. 

Much has been learned during this first phase regarding the development of vision statements/ 
service roles, performance measures, MIS, and planning strategies in academic health science library 
settings. Indeed, one of the major findings from the first phase is the need for AHSLs to think in terms 
of a total management system that includes strategic planning, performance measurement, and 
management information systems. But the major challenge that remains is to link and integrate these 
activities into a comprehensive management approach. AAHSLD can be the catalyst to develop such a 
management approach. 

While we can note that much has been learned during this first phase, we must also note that much 
research and development work remains to be done. As detailed in the appendices, a number of key 
research questions regarding the nature of AHSL institutional settings, defining and measuring impacts 
and outcomes of AHSL on the larger institution and user community, and determining how best to define 
and collect data appropriate for the national database (Annual Statistics) versus that appropriate for 
the local AHSL (to name a few), will require serious attention and investigation. 

At project completion, AAHSLD will have served, as the catalyst to produce a practical and 
innovative management manual, a program of workshops and seminars for utilizing the manual, and a 
redesigned Annual Statistics database and .annual report. These accomplishments will provide a 
framework for health sciences libraries to grow and successfully meet the challenges of the 1990s. We 
look forward to working with AAHSLD on defining this project in greater detail and bringing it to 
fruition. 
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APPENDICES 

A. Planning, Performance Measures, and Data for Decision Making: Key Sources and 
Issues for Academic Health Science Libraries (submitted separately to AAHSLD) 

B . Review of Academic Health Sciences Library Planning Documents 

C. Summary and Analysis of Academic Health Science Library Focus Groups 

D. Assessment of Needs and Priorities for the AAHSLD Statistical System 

E. Summary and Analysis of Medical Center Administrators Focus Group 

F. Vision Statement Critique Exercise (used at the November 9, 1991 workshop) 

G. November 9, 1991 Workshop Schedule and Group Exercises 

H . Summary of Planning, Service Roles, Performance Measures, and MIS for 
Academic Health Science Libraries Workshop 
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REVIEW OF ACADEMIC HEALTH SCIENCES LIBRARY 
PLANNING DOCUMENTS 



A Draft Report Prepared for 
AAHSLD Editorial Board 

By 

Charles RMcChire and Nancy A. Van House 
For the Study 

A PLANNING, SERVICE ROLES, AND PERFORMANCE 
MEASURES MANUAL FOR ACADEMIC HEALTH SCIENCE LIBRARIES 

September 26, 1991 



Purpose of the Report 

As one step in the data collection for this study, a number of existing planning documents from 
academic health sciences libraries were examined by the study team. These documents included 
mission statements, strategic plans, and statements of goals and objectives, and ranged from one-page 
preliminary statements to extensive and 5- year planning documents. 

Thirty-three planning documents were submitted in response to a letter from the Association of 
Academic Health Sciences Library Directors. The libraries submitting plans represented a range from 
small to large in terms of size of institution, number of primary clients, and total number of volumes.! 
The plans were examined specifically to determine how well they meet generally agreed-upon criteria 
for what constitutes an effective strategic plan. 



Criteria Used to Assess Plans 

The specific criteria used for evaluating the planning documents are presented in Figure 1. This list 
is based on various models of strategic planning and represents a generally accepted set of criteria for an 
effective approach. Plans were evaluated in the following broad areas: 

A. Purpose and Philosophy 

Does the planning document identify the library's aims in planning? Articulating aims helps to 
focus the planning process and enables the library to evaluate the process at the end of a cycle: were the 
aims met? 

■ Describing the conceptual approach to planning also helps to underscore tile library's aims, as well 
as explaining the ways in which goals, objectives and strategies are defined. Whatever the approach, 
effective strategic planning involves a commitment to staff and constituent involvement, identification 
of mandates from the university or other parent organization, and a specific time frame with provision 
for revision of the plan and the beginning of a new planning cycle at the end of this period. 
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FIGURE h CRITERIA FOR EVALUATING FLANS 

A. Purpose and philosophy 

1. identifies purpose and aims of planning 

2. describes conceptual approach to pUnning 

3. commitment to staff/constituent involvement 

4. identifies mandates from parent organization, state agency, etc. 

5. covers a specific time period, with provision for revision and 

for beginning a new cycle 

B. Environmental/Needs Assessment 

1. planning proem includes assessment 

2. employs formal information collection and analysis 

3. needs of user group are identified 

4. internal and externa] environments are examined 

5. library's strengths, weaknesses, opportunities, and threats are 

identified 

C. Mission and Roles 

1. planning document includes a mission statement 

2. mission statement is a concise statement of library's purpose and 

of service roles chosen for emphasis 

3. mission statement includes an expression of library's values 

4. makes a distinction between primary and secondary service roles 

5. selects a limited number of roles, rather than attempting to meet 

all needs of each user group 

D. Goals, Objectives, and Strategies 

1. articulates goals: long-range, broad statements describing a desired * 

condition toward which the library will work 

2. relates goals to mission and to needs assessment 

3. objectives are do-able, measurable, time-limited targets contributing 

to goals 

4. prioritizes goals and objectives and selects most important 

5. selects a limited number of goals and objectives for attention 

during the current cycle 

6. frames specific strategies (policies, programs, services, actions, 

decisions, or resource allocations) to achieve objectives 

7. identifies and compares alternate objectives and strategies 

8. arranges objectives and strategies by department or library unit 

E. Strategic Nature of Plan 

1. plan identifies specific strategic issues 

2. goals, objectives and strategies deal with strategic issues rather than 

with regular, ongoing operations 



2 



25 



McOure/Van House 



AAHSLD Study 



Draft, September 26, 1991 



FIGURE 1: CRITERIA FOR EVALUATING PLANS (CONTINUED) 

F. Implementation 

1. implementation is a component of formal planning document 

2. schedules tasks and responsibilities and assigns them to individuals/ 

groups 

3. identifies resources (staff, equipment, finances, etc.) needed for 

implementation 

4. identifies implementation budget or discusses budgetary impacts 

5. implementation includes formative evaluation 

G. Evaluation 

1. evaluation is a component of formal planning document 

2. criteria are established by which success of activities can be assessed 

3. defines specific performance measures for evaluating extent to 

which the library fulfills roles, achieves goals 

4. identifies impacts (outcomes) on environment 

5. relates evaluation to needs assessment 

6. focuses on outputs/effectiveness not inputs/efficiency 

H. General Principles 

1. involves assessment of users and env ir on m ent 

2. alternatives are identified and compared 

3. a limited number of items for action are selected and prioritized 

4. focuses on action rather than planning 

5. targets strategic issues 

6. identifies resources needed 

7. provides for evaluation, determination of library's success in 

meeting goals 



B. Environmental/Needs Assessment 

Strategic planning requires the collection and analysis of information about a library's internal 
environment, its external environment, and the needs of its users. Without such information, the 
planning process will fail to respond to important realities. Strategic planning is impossible without 
an understanding of the library's strengths and weaknesses, the opportunities and threats presented by 
its environment, and the needs of its various user groups. A variety of approaches have been developed 
for obtaining such information, including staff discussions, open meetings, end formal data collection 
(e.g. surveys). 

C Mission and Roles 

A mission statement "expresses briefly and directly the library's purpose and service priorities for 
the current planning cycle."* It should be concise and clear. Often, the mission statement also 
emphasizes an organization's values, i.e. the principles that guide the organization as it pursues its 
mission. 
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Roles are profiles of service areas which a library chooses to emphasize. Each role is a shorthand 
way of identifying a group being served, the services being provided, and the resources needed to 
achieve these ends. The mission statement articulates the roles selected and builds upon them. 

Developing service roles encourages a library to examine its various user groups and their needs, to 
anticipate future trends and needs, and to set priorities. Since no library can function equally well in all 
areas for all people, a limited number of roles should be chosen for emphasis, and a distinction should 
be made between a library's primary role and secondary roles. 3 

D. Goals, Objectives, and Strategies 

These terms are used with a variety of meanings in different planning approaches and planning 
documents. They are even used interchangeably. Most approaches, however, agree that the planning 
process is a "funnel" that moves from broad aspirations to specific actions. In keeping with that model, 
the following definitions are appropriate: 

Goals are long-range, broad statements that describe a desired future that the library is working 
towards. Goals should be related to the library's mission, to its users and environment, and to strategic 
issues which the library has identified. Goals phrased in terms of outcomes rather than actions will 
lend themselves to more effective evaluation (i.e. "Users are able to obtain timely, accurate 
information" rather than "The library provides timely, accurate information to its users"). 

Objectives are short-range and "describe the results to be achieved in a specific time period."* That 
is, they are specific targets that the library aims to achieve, and are directly related to goals. They 
are not lists of activities or programs that support a goal. One example of an objective for the above 
goal would be To increase the percentage of correctly answered reference questions to 95% by the end of 
1991." Objectives are do-able, measurable, bound to a specific time period, and usually begin with an 
action verb. 

Strategies are programs, approaches, or activities developed to meet a given objective. Strategies 
may be general rather than specific, and a given strategy can be pursued by any of a number of different 
actions. One strategy to meet the above objective would be to "increase the percentage of reference desk 
hours during which professional staff are available for reference service." This could be done by hiring 
additional staff, by reallocating staff responsibilities, by limiting reference hours to times when 
professional staff are available, etc. 

It is possible to generate a number of objectives to support each goal, and a number of strategies to 
achieve each objective. For this reason, it is important to (1) identify and compare alternatives, 
selecting those which have the highest likelihood of succeeding (while balancing other factors such as 
risk and cost), (2) select only a limited number for implementation, and (3) prioritize those which are 
selected, so that the most important are pursued even if others Stave to be sacrificed. 



.£. Strategic Nature of Plan 

Strategic planning is not an all-encompassing process that comprehensively details every aspect of 
a library's operations. Rather, it is a process which identifies specific issues that a library faces 
(based on its strengths and weaknesses, and opportunities/threats present in its environment) and 
responds to them. Bryson (1988) defines strategic planning as: 5 
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• disciplined effort to produce fundamental decisions and actions that shape and guide what an 
organization (or other entity) is, what it does, and why it does it At its best, strategic planning 
requires broad-scale information gathering, an exploration of alternatives, and an emphasis on 
future implications of present decisions. It can facilitate communication and participation, 
accommodate divergent interests and values, and foster orderly decision making and successful 
implementation. 

Strategic planning differs from traditional long-range planning models in a number of ways.-* 

1. It relies more on identifying and resolving issues, rather than simply identifying goals and 
objectives and fitting them into current budget and work programs. 

2. It emphasizes assessment of the environment outside and inside the organization. 

3. Strategic planning generates a "vision of success" (an idealized image of the organization) and 
ask how it may be achieved. Strategic plans often represent qualitative changes, while 
long-range plans may simply project existing trends into the future. 

4. Strategic planning is more action oriented. 

Central to this approach is the identification of strategic issues 7 

fundamental policy choicefs] affecting an organization's mandates, mission, values, product or 
service level and mix, clients or users, cost, financing, organization, or management." 



F. Implementation 

Effective planning does not stop with the development of goals, objectives and strategies, but also 
provides for their implementation. This requires scheduling specific tasks and assigning them to staff, 
identifying and securing necessary resources, and setting up a process of formative evaluation to keep 
the implementation process on target and indicate areas where revision of the plan is necessary. 

G. Evaluation 

Finally, effective strategic planning provides for the evaluation of strategies and activities, to 
determine whether or not the library is meeting its goals. Evaluation must be provided for early in the 
planning process, by phrasing goals in terms of outcomes (impacts on the environment) and by 
developing measurable objectives. The planning document should provide a sense of (1) which criteria 
must be met in order for activities to be considered successful, and (2) what specific performance 
measures will be used to assess whether activities meet these criteria. Lastly, the library must plan 
the way in which evaluation will be carried out, assigning specific tasks and time frames. 

Each library plan was assessed on criteria in these seven areas, plus a number of "General 
Principles" (important concepts which cut across the different functional areas of the planning process) 
as defined in Figure 1. 
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Assessment of the Plans 

The plans received varied greatly in purpose, length, level of detail, and level of completion. 
Some seemed like preliminary sketches for a planning process, and a few Indicated that they were 
indeed preliminary in nature. Plan G, for instance, consisted only of a mission statement; the library is 
currently beginning a planning process. Plan C included earlier lists of goals and objectives with a new 
proposal for a planning process. Where appropriate, the code Va" is entered to indicate that the plan 
cannot yet be evaluated on certain criteria. 

Several plans are descriptive rather than strategic. That is, they are essentially intended to 
document what the library is, what it does, and perhaps what it aspires to be and do. Although such 
plans often contain interesting mission statements, they usually culminate In exhaustive lists of goals 
and tasks which are little more than "wish lists" of everything the library would like to be able to do. 
Although organized into "goals" and "objectives," such lists are of little use in strategic planning. 

Many library plans were developed as part of a system-wide or University-wide planning process. 
Plan C began at the very top with state agency mandates. In these cases, it is difficult to assess the 
library's approach separately from that of the overall organization, and the matrix ratings reflect a 
consideration of both levels. Plan C, in particular, was difficult to assess, since it stops at the 
University level and has not yet been detailed at the library level. 

Figure 2 presents a rating of each plan on the criteria described above. The following symbols are 
used: 

0 The plan fails to meet this criterion 
+ The plan meets this criterion 
++ The plan excels in this area 
n/a Not applicable for this plan 



Discussion of Findings (refer also to Figure 2) 
A. Purpose and philosophy 

While several of the plans included a discussion of purpose and philosophy, many do not. The 
majority of the planning documents begin with a mission statement, with no preliminary discussion of 
the planning process. These plans, which provide no indication of the aims of the planning process, 
usually also lack any provision for implementation and evaluation. They appear to be descriptive in 
intent rather than to be plans for action. 

Plan K, however, stated two specific aims for the strategic planning process: 

1. Prepare a strategic plan and use as the basis for resource allocation and other decisions. 

2. Derive a detailed annual operating plan from the priority areas of the strategic plan. 

Plan D provided a concise, clear statement of the purposes behind planning, defining "strategic 
planning" and presenting three broad aims and seven specific objectives for their particular planning 
initiative. The objectives cover a wide ground, including some points often assumed but rarely 
articulated: 
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1. Present where the library is going and how it will get there 

2. Review current activities so as to make proper adjustments and modifications 

3. Know where to concentrate resources 

4. PR tool for use with university administration, faculty and students 

5. Improve communication within the library 

6. Provide answers to the hard questions on space, resources, use of technology 

7. Gain knowledge about the library's strengths, weaknesses, threats and opportunities 

Although several of the plans did cover a specific time frame (typically three to five years), only 
a few provided for what would happen at the end of that time in terms of revision and the planning of 
a new cycle. 

Plan O was among the more sophisticated: four-year plans are developed every two yean, with 
the overlapping period revised as it becomes current (for example, the last two years of the 1990-94 
plan will be revised in 1992, as implementation becomes imminent). Plan B' is revised annually. The 
administration sets library priorities each year and each department submits bimonthly objectives 
detailing how they will fulfill the library's mission and goals. A progress report is issued every four 
months. Plan R demonstrated a yearly objectives cycle: the list of goals and objectives from the initial 
plan is paralleled but revised and expanded in each year's annual report, with specific objectives 
selected for the coming year. 



B. Environmental/Needs Assessment 

The degree of environmental assessment and user needs analysis incorporated into the plans varied 
greatly. Many plans began with a mission statement and proceeded to goal development with no 
situational assessment whatsoever. Others included environmental assessments or lists of strengths and 
weaknesses, often generated by library administrators or planning committees. Sometimes the planning 
process included assessment, but the assessment was not discussed and analyzed in the planning 
document. Only a few plans actually incorporated an exploration of user needs. 

Plan D, for instance, contains an extensive list of strengths and weaknesses under such categories as 
"Personnel" and "Collections." Unfortunately, this assessment was entirely carried out by "middle and 
upper level library administrators;" planning was a "top-down process" and other librarians and staff 
were not involved until a draft had been prepared. There is no indication that user input was sought at 
all. 

Plan T included an extensive assessment of library staff, conducted during a retreat. Categories 
included values, concerns, hopes, and demands made by users, administration, technology, the 
profession, and staff themselves. 

Plan A was one of the few to contain an extended assessment of users and environment A number of 
important statements were made in this section, e.g. "If reduction in collections, services, or efficiency 
occurs, there will be increased pressure for departmental libraries resulting in duplication of collections, 
services and competition for funds." Unfortunately, mere is no documentation of how this assessment 
was conducted, and it seems that these are simply "intuitive" needs identified by the planners with 
little formal information gathering. 
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Plan J included an extensive environmental assessment which examined and predicted the impact of 
advances in information technology and of economic factors on the library's future. It also compared the 
library with its peer group, and was one of only two plans to make reference to the set of standards 
developed by the Association of Academic Health Sciences Library Directors and the Medical library 
Association. 1 

Plan A' featured a lengthy environmental scan examining trends (and impacts on the library) in the 
following categories: Academic and Research Clinical Issues, Information Technology, Societal Factors, 
Economic Factors, and Library Factors. Plan B' examined Personnel, Financial Resources, 
Sodo-Political, and Organization and Management in a similar way. Plan V also included a lengthy 
section discussing future trends in libraries and publishing, current (internal) resources, and changes 
needed. In most cases, no supporting evidence or literature was cited for these discussions of trends and 
impacts. 

Plan O included a large appendix of statistics, but these are not discussed or incorporated into the 
plan (even such striking figures as a 51% drop in circulation over a period of five years, or a 167% 
increase in unfilled ILL requests during the same period). Plans R and V also included such basis 
statistics. 



Plans F and F were the only ones to include an instrument for formal data collection. Plan F 
distributed a six-page library evaluation survey to faculty, staff and students. The cover letter to the 
survey stressed that the form should take less than ten minutes to complete. The survey mentions 
several possible future developments for the library (new services, broader access, increased library 
instruction) and asks the users how frequently they would use each potential development. Also, in 
asking users to rate current library collections, services, facilities, etc., the survey asks not only for the 
user's level of satisfaction, but for a rating of the importance of that particular item. This data could be 
particularly useful in planning: how worthwhile is it to do something very well if no one considers it to 
be particularly important? This survey is a good example of a straightforward, uncomplicated 
approach which can gather a wealth of useful information. 

Flan F used a similar instrument, although this was longer and asked for some elaboration in terms 
of what the library should offer and what the library's role should be. The survey also asked those 
who don't use the library to explain why, and to describe where they do get their information. 

It is important that a plan's assessment focuses on the needs of its users and not hist on the needs of 
the library. Plan M presents a section on "Current Realities" which is an extensive discussion of what 
the library needs which the university has not been providing. While this is important, by failing to 
tie these needs to user services and benefits the library diminishes its arguments and produces a 
planning document which reads primarily as a request for more nioney. 

As mentioned earlier, several plans were part of system-wide or university-wide planning 
initiatives. One difficulty with such plans is mat they tend to respond to the mandates and directions 
provided by the parent agencies to the exclusion of assessing the needs of the library's own users. 
Following the game plan takes the place of environmental and needs analysis. Libraries need specific 
information about their users, and this information is different from that required for planning ».t 
higher administrative levels. When a library is part of a university-wide process, they must be 
allowed and expected to engage in their own information gathering activities. 
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C Mission and Roles 

The concept of service roles Is not explicitly used by any of the plans examined. Several plans, 
however, implicitly identify roles in their mission statements or elsewhere. Han B, for example, 
identifies "Dements of Distinction" several of which could be taken as roles: 

• Reputation as a proactive, client-oriented information agency 

• Campus-wide recognition for quiet, comfortable study areas 

• Significant regional information resource for health care and human service clinicians 

This section is meant limply to highlight "the comparative advantages of the unit," and is not 
explicitly taken into account during subsequent planning steps. 

Plan K provides a good example of an implicit use of roles. The library's mission statement 
indicates that the library "is dedicated to excellence and leadership in four areas." 

1. Serving as a leader in information management at [the university) 

2. Providing excellent health information services 

3. Providing health Information resources 

4. Becoming a national center of excellence in information management research 

There are three roles expressed here: (1) leader in information management at the university, (2) 
provider of health information, and (3) information management research center. 

Library values are sometimes implied, sometimes articulated. Plan A identified a number of 
"Guiding Principles" in various areas of library activity. Plan B contains an excellent statement of 
library values, under headings such as Heritage, Ethnic/ Gender/Cultural Diversity, and Faculty and 
Staff Standards. Plan K described two "recurring themes:" 

• Service - supporting access to information, document delivery, consultation, quick answers and 
a host of related activities. 

• Advocacy - functioning as a proponent of information literacy, strategic planning and 
management of information resources. 

Plan K also described five "enabling mechanisms" necessary for the achievement of goals and 
objectives: Facilities, Finance, Human Resources, Planning, and Systems. No specific requirements in 
any of these areas are provided. 

To a large degree, the libraries indicate a perspective of "trying to do all things for all constituency 
groups all the time." Some of the documents contained ideas for service roles that might help the 
library set priorities for what activities are most important These ideas (and others from other data 
collection activities) are being brought together by the study team as a menu of possible service roles for 
academic health science libraries. 
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D. Goals, Objectives, and Strategies 

Although most libraries did identify goals and objectives, there was no apparent effort to 
prioritize objectives and strategies, nor to select a tew for attention rather man trying to "do it all." On 
the contrary, most plans started with so many high-order goals that by the time objectives and 
strategies were developed for each of these goals an exhaustive, impracticable list resulted. In Ran B, 
the library's administrative office alone identified 25 objectives for a three-year planning period. 

Several plans developed goals by articulating one goal in each of several topical or functional 
areas (for instance, in Plan Q: Access, Education, Cooperation, Innovation, Staff, Collections). The 
result of this approach is that goals, rather than addressing specific issues or aspirations, tend to be 
variations on "We are going to do Access (Education, Cooperation, etc.) very well." They are too broad 
to be of much use in providing direction for the library's activities. 

No plans included Objectives as defined earlier in this report: do-able, time-baser;, measurable 
outcomes which serve as targets to support goal fulfillment. Most plans began with goals, divided 
these into sub-goals called "objectives" (but which are still broad statements), then proceeded directly 
to identifying strategies or programs (sometimes also called objectives, i.e. Plan D: "objectives: specific 
actions that will carry out the goals") to achieve the goals. By failing to set target, objectives, these 
plans fail to establish a link between strategies and goals (i.e. how will » given program contribute 
towards a goal?) and fail to establish criteria by which success can be quannfiably Measured. 

Plan I approaches this definition of objectives: 

• Implement 24 hour turnaround service for intermediary online literature ?«arch service (for the 
goal: "Improve library service to users") 

• Add at least 75 new journal subscriptions (for the goal: Increase library resources") 

Although these are not time-bound, they are measurable. The first one is particularly useful as an 
objective in that it (1) obviously supports the goal, and (2) could be achieved through any of a number of 
different strategies. 

Actually, several plans contained good definitions of just what an objective should be, but few 
planning teams seemed able to write objectives that fit these definitions. Plan B, a university-wide 
initiative, features a set of guidelines for the planning process. These guidelines provide useful 
instructions for writing objectives: 

objectives should be simple and directly related to the MOST important goals of the planning 
unit ... The number of statements should be limited to those that can be addressed effectively 
during the planning period. It is not necessary or even desirable to develop a comprehensive set 
of objectives that describe every detail of operations 

The plan, however, does not follow these guidelines. 
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Few plans prioritized goals to any extent, or examined alternatives. Plan J identified "Priority 
Jj 0 ^'™ a " d New Initiatives;" four areas of greatest importance for the next four years. Once 
identified, these areas are referred to later in the pUn as "the most important new programs" and 
other areas are considered secondary. The plan also includes a section on "Essential wigotog ronctions" 
and one on ^xpanded features of less urgency." Plan Y stated that the objectives listed were "selected 
from a broad list of possible objectives" and "are those identified as most important to achieving the 

Plan C, which was more of an analysis than a plan, asked for alternatives and contingencies: Is 
there another way to do this? What would happen if it's not done, or is cut back? 



E. Strategic Nature of Plan 

As mentioned earlier, strategic issues were sometimes described in the midst of a plan without brine 
specifically identified as such. A number of plans specifically addressed strategic issues factor the 
library. On the other hand, some plans which called themselves "strategic" did not deal with issues 

At Jill* 

The background information provided with Plan D, for instance, indicates that "critical issues were 
identified before the mission, goals, and objectives were written." Unfortunately, these issues are not 
articulated anywhere in the planning document, and the goals once again appear to address the entire 
range of library activities, rather than specific issues. 

Plan J does identify a number of strategic issues in its environmental assessment (e.g. expansion of 
telecommunications networks, growth of electronic media), but does not specifically develop eoals that 
respond to these issues. ' re 

Plan X identifies critical issues within each goal area, and specific actions to address each issue. 
Although some of these "issues" are very specific needs (e.g. for a specific new piece of office 
equipment), others are strategic in nature (both internally and externally focussed). The development 
of the plan revolves around these issues. 

PUn F identified ten environmental factors with impact on the library, which require a planned, 
strategic response. Goals, objectives, and strategies are specifically developed for six key issues. 

There are ways to incorporate important ongoing activities into a strategic planning document 
without sacrificing the issue-based nature of the process. Plan J simply incorporates a heading for 
"Essential ongoing functions" and states that Certain primary functions of the library must be 
maintained and the quality of service kept at a high level." 

In general, although a number of plans identified strategic issues, far fewer actually addressed 
these issues with goals, objectives, and strategies. Also, there, seemed to be a lack of connection between 
environmental assessment and issue identification; some libraries conducted assessments but then did 
not explore the issues raised, while others boldly stated critical issues without any documented 
information-gathering activity. 
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F. Implementation 

Many plans, chiefly those of a descriptive nature, made no provision for implementation. Few had 
specific tasking or steps to implement the plans. Such plans seem primarily intended as lists of what 
the library aspires to be and do, rather than strategic guides to action. 

A few plans approximate scheduling without really doing it Plan B, for instance, establishes a 
time frame" for each objective. The time frame is gr* ^ally the academic semesters) during which 
the task will be carried out. There is no target for completion, only t vague notion of when something 
will be happening. Plan D breaks strategies into -year 1, Year 2, Year 3," but provides no specific 
timetable for the accomplishment of these activities. 

Only a few plans (e.g. Plans L, Q) assigned specific activities to specific individuals or groups for 
implementation. This indicates an orientation towards action, and the "Progress Report" included with 
Plan L indicates that most of its objectives have been accomplished or are underway. Plan Q uses Action 
Hans: "specific activities, for which responsibilities and resources can be assigned and ... a target date 
can be established." Target date and individual^) responsible are indicated for each action plan. 

Plan Z includes lists of measurable "support activities" for implementing objectives, with "success 
criteria," "responsible people," and "time." Although this only approximates the use of performance 
measures for developing objectives, it does specify an implementation plan and provides for 
determination of its success. 

Plan B' used an approach to implementation that anticipated difficulties. For each action step, 
problems were anticipated and solutions developed to eliminate or lessen the impact of these problems. 
These solutions are then worked back into the original sction plan as possible. In this plan, tasks and 
activities are given due dates, assigned to people, and tied to specific goals. 

Even those plans which did provide for implementation often did not incorporate any discussion of 
the resources which would be needed, or the overall budgetary implications of the plan. The many long 
lists of goals, objectives and strategies would have required trimming down if financial realities were 
considered. Such a discussion would have encouraged prioritization and selection; such choices will 
have to be made anyway in the course of implementation. Plan B, for instance, indicated that 
"strategic budgeting will follow strategic planning." While this is understandable, linking the 
processes more directly together keeps the strategic plan more realistic by including considerations of 
cost-effectivsness and cost-benefit from the very beginning. 

Only a few plans made any specific provision for budgeting. The system-wide library plan for Plan 
D described resources needed for each year of the plan (1-3) to accomplish each goal. A "Summary of 
Resources Needed" provided specific figures for each year, in FTE for staff and dollar amounts for other 
categories. Plan L, a library-specific plan, provided specific cost figures (either one-time or per year) 
for most of its objectives. Plan V contains both a discussion of resources needed (including rationale for 
need) and an estimate of program costs. An appendix to the plan attempts to relate personnel needs to 
specific programs. 

Plan M contains a detailed consideration of resources needed for implementation, although it does 
not provide for the implementation itself. A section on "Resource Priorities for New Funding" identifies 
"priorities for additional funding to support programs and activities reflected in the strategic plan." 
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Detailed budgets of recurring and nonrecurring resources are Identified for each year. Although each 
yearly budget begins with a summary of what the funds will be used for {i.e. the goals/objectives/ 
?*! e f i f , i°7 Uut 11 • eem » «*»t the strength of the argument for addirional resources is 
z!!l?^ hed J S y l no1 d,rectly ***** "q u «*» to •Pedfic pUns and specific outcomes. What, 
precisely, will the money be used for? The reader has to make the connection, and may be more likely 
to simply react negatively to the amounts called for. 

A few plans also included a discussion of current resources. Plan Z, for instance, used a matrix to list 
programs (ranked in importance), existing resources (ranked as No, Inadequate, or Adequate), and 
additional resources needed. 

An interesting aspect of Plan Mis the inclusion of "Financial Scenarios," discussing contingencies for 
reacting to cub or increases in the budget The plan specifically indicates what actions would be taken 
in these cases, and what the results would be. Plan O also discusses the impact of budget reductions 
and projected reductions. 

Some plans intentionally avoided any discussion of resources. Plan Y, for instance, Is not limited 
by definitions of what could be accomplished with current resources, but rather states what should occur 
if the library is to fulfill its mission." While this refusal to compromise is commendable, it is unlikely 
that the steps outlined in the plan will occur without a consideration of available resources and 
resources needed. 



Very few plans made any mention of formative evaluation. Plan Y simply states that "there will 
be periodic evaluations of progress in accomplishing goals and objectives." Plan N makes specific 
provision for formative evaluation in the form of "Product Evaluations," ongoing evaluations of the 
services and functions that "translate" the library's mission. Three products are considered, each with 
several subproducts: Collection, Delivery of Information, and Creation of Information. Regular 
evaluations of each subproduct are conducted. 



G. Evaluation 

As with implementation, only a few plans made any provision for evaluation. Those plans that 
did used a variety of approaches, but none focused on specific, measurable outcomes. 

Several plans equated "documentation" or "progress" with evaluation. Documentation, however, 
merely provides proof of the fad that something happened, and perhaps the extensiveness and 
efficiency of an activity. It does not in any way measure the impact which an activity has had. For 
example: 

• Plan B described "Evaluation" and "Results" for each objective. But the evaluation is often 
merely documentation of the activity, and "results" often refers to the simple outcome of the 
activity rather than its impact For instance, for the objective "Recruit and hire an education 
librarian" the following are provided: 

Evaluation: Maintain a file documenting advertising recruitment efforts, application* and 
recommendations for hiring. 

Results: Reach agreement with candidate for position, negotiate starting date. 
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• In Plan C the "Measurement Procedures" indicated for each objective /ire usually "annual 
reports" or other reports documenting activity. 

• Plan J began with an "Assessment of Progress" which documented recent acromplishments but 
did not evaluate their effectiveness. Likewise, Plan O reviewed the library's success/failure 
in accomplishing the previous year's goals. 

• Plan L included a "Progress Report" which indicated the degree to which each objective had 
been implemented, but did not examine their impact on the environment 

Plan C also included "Outcome performance indicators" for each objective. While these did 
generally focus on outcomes, as criteria for evaluation tiny were vague and open-ended, with no specific 
targets set Some, like "Number of graduates who practice [in-»tate]", could have easily been named 
into useful measures if only the plan had indicated a specific target number or a specific increase from 
current levels. 

Some plans did identify desired outcomes on the environment, even if evaluation was not a formal 
part of the plan. They framed goals and objectives in terms of outcomes rather than in terms of things 
the library should do. It would only be a small step from here to developing specific performance 
measures for assessing the degree to which these outcomes are achieved. 

The case study article accompanying Plan Q indicated that the planning process led to an 
understanding of the need for both evaluation and research, and made several specific suggestions for 
action research that could be carried out by the library to provide information for planning and 
decision-making. 



H. General Principles 

The underlying model in many of these plans is that of the library which does everything well for 
everybody. Assessments were often phrased as "continued demand," or "increased emphasis needed," 
etc. with little consideration of which, if any programs could be cut. Tne Annual Report submitted 
with Plan L states: 

Armed with a new long range plan, library staff enthusiastically accepted the challenge and 
initiated many new programs while maintaining the old. 

Also Plan P: 

The vision of the library is to become all things to its users within the context of meeting expressed 
information needs. 

The idea of T>eing all things to all people" differs with much of the current literature on missions and 
service roles for libraries. 

Prioritiziation was not a component of most plans, either at the level of mission/roles or at the 
level of specific objectives and strategies. A campus newspaper article publicizing Plan C stressed that 
the purpose of planning was to set priorities, that "universities can no longer be all things to all 
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people ." The plan itself, however, still reads like a wish list of everything a university should be. If 
priorities are established, they are not articulated, and there is no sense of anything being given up. 

The majority of plans were action-oriented rather than "planning for the sake of planning." Plan X, 
for instance, even included a separate list of "internal actions" to be addressed internally without need 
for further elaboration in the planning document 

Other basic components of strategic planning were absent from most plans examined: generation and 
examination of alternatives, targeting of strategic issues, and consideration of resources needed. 
Rnany, although the libraries had obviously put a great deal of time and effort into the planning 
process, none of the plans effectively provided for determining whether or not the library succeeds in 
meeting its goals. 



Assessing Che Plans as Written Documents 

In addition to the criteria for effective planning discussed above, there are other important 
characteristics of the planning documents themselves. Are the documents well -organ! zed and clearly 
presented? Are they written in a style which is generally comprehendible, with a minimum of library 
jargon? The thirty-three plans submitted by academic health sciences libraries were examined in light 
of these criteria, as well. 

In general, the plans were well-written and easily understood. Those which were intended mostly 
as internal documents differed in style from those intended for a larger audience, but in both cases the 
style was clear and usually concise. 

All plans were clearly organized into identified areas discussing Mission, Goals, Objectives, etc. 
Some version of an outline form was generally used. Beyond this, however, the plans varied from 
bare-bones outlines to long, wordy documents. Goals and objectives, for instance, were usudly expressed 
as single sentence statements but occasionally became long and discursive. In Plan E, objectives were 
written out in paragraph form, often rather long, containing several different points. Keeping goals and 
objectives as simple statements avoids the following pitfalls: 

• writing long descriptions which are difficult to read and obscure the main point, 

• using a long description to avoid stating specifically and upfront what the goal or objective 
actually is, and 

• combining several goals or objectives into one statement, disguising the fact that more than one 
idea is being expressed. 

At the other extreme, plans which were simply outlines of goals and objectives left many questions 
unanswered: the aims of the planning process, the approach used, and ways in which implementation 
and evaluation would be addressed. 

Goals and objectives were written at varying levels of specificity. Within the same planning 
document, some goals might deal with broad, long-range growth while others deal with the 
implementation of one very specific program. Although some variability is unavoidable, this range of 
specificity can be confusing both to the reader and in terms of implementation. 
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This problem could be helped if terms wen as "goal" and "objective," as used in the planning 
process, were defined within the document (some plans did this, for example Plans B and D). This 
would provide written goals and objectives with a framework for consistency. Plan I demonstrates what 
can happen when terms are rat defined: the overall J-year plan states only one goal and supports it 
with seven objectives, but in the one-year detailed plan, these same objectives are labelled as goals. 

Some plans included a general long-range plan followed by either a one-year or a departmental 
plan derived from it. Sometimes there was a lack of agreement between goals as expressed in the larger 
plan versus the more detailed plan. When there is no explanation of this, confusion arises as to which 
is the "correct" version. 

Several of the plans incorporated interesting ideas for presentation: 

• Plan M presented a "Summary of Goals" before proceeding to the detailed outline of 
goals/objectives/strategies, allowing the reader to get the "broad sweep" of goals articulated 
by the library. 

• Plan J began by reviewing past accomplishments (from previous plans) and current status, 
providing some concrete examples of what has been done before moving on to what will be done. 

• Plan K succinctly and effectively expressed its mission by briefly identifying four "areas of 
excellence and leadership," two "recurring themes" of the library's goals (service and 
advocacy), and five "enabling mechanisms" necessary to achieve goals and objectives. 

• Several plans ( J, M, V) compared the library with its peer group on a number of criteria. While, 
this does not necessarily contribute to the library's needs assessment, it does provide a broader 
context for discussion. Such comparisons are also used as arguments for improved funding and 
facilities. 

• Plan B, a university-wide planning initiative, was a good example of moving from high 
administrative levels to specific units. Eight basic goals were addressed at the university 
level. The health sciences library developed a unit plan to respond to these, and then each 
individual department of the library developed its own operational plan for the three-year 
period. 

• Plan Q included cross-references between related goals and objectives. 

• In order to emphasize the need for additional resources, Plan X presents "Goals and Objectives 
Requiring Financial Support" before those "Requiring No Financial Support." 



Key Issues and Recommendations 

Several key issues for academic health sciences libraries can be identified from this examination of 
planning documents. 

1. How can planning data be collected, managed, and used m the decision making process? 

The plans examined make little use of information^athering, although such information is critical 
for effective strategic planning. Even plans which incorporate environmental assessment appear to 
base this assessment on limited information. By failing to make use of planning data, libraries can fail 
to respond to important needs, opportunities, and threats. 

Recommendations: 
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• ^^Z^^^ elt Wld t0oh th * t Ubf * rie * an u * *«> information on their 

• Develop approaches that libraries can use to examine this information to identify streneths 

?T^* 9 !2l2 ,p0rtUnit,e *' thpeito ' "* to «»ly» these to identify strategic issues' 

• Develop effective management information systems to support this process. ^ 

i How will the libraries know when they have accomplished goals and objectives? 

* on ; P I,n ? provided "*thods for determining whether or not a library has achieved its 
g^handobjectim. taorder to evaluate library performance in terms of actual outeoir« ^ im%S 
on its users, specific performance measures are needed. Also required is a planning aroroach^ch 

'J^^r^^J 1 ^ ^ oi P™«* » the Jde^oTrS^^ 

development of goals and objectives, and in the implernentition of programs. 

Recommendations: 

• Phrase goals in terms of actual outcomes/impacts rather than in terms of library activities. 

• Identify specific, measurable objectives rather than broad categories of activity 

• Develop specific performance measures that can be used to assess the degree to which objectives 
and goals are met Performance measures must focus on quantifiable outcomes, not on library 
"inputs" (budget, acquisitions, staff) or on accomplishment of tasks. 

3. How can library plans be made more realistic and strategic? 

Some of the plans examined were vague and very not practicable, and did not provide for 
imp ementation of strategies and activities. Most avoided the question of resources needed to 
implement the plan. Few of the plans were strategic in the sense of responding to specific, critical 
issues confronting the library. 6 ^ ' 

Recommendations: 

• Begin the planning process with an identification of planning aims, and an extensive assessment 
of users and environment. Develop goals that respond to specific issues identified in this 
process. 

• Develop an approach to strategic planning for academic health sciences libraries which 
ac^owledges the need to make tough choices rather than attempting to "be all things to all 

• Develop specific service roles for academic health Sciences libraries, which can be used by 
these libraries to focus their planning efforts. 

• Incorporate the principles of (1) generation and examination of alternatives, (2) selection of a 
limited number of items for emphasis, and (3) prioritization into academic health sciences 
libraries planning. No list of goals, objectives or strategies is useful until it has been 
pnoritized. 

• Develop a planning approach that helps libraries "bridge the gap" between preliminary 
planning (generation of goals etc.) and implementation. Develop approaches and took for 
tasking, scheduling, budgeting, and formative evaluation (feedback). 
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4. What art the issues facing academic health sciences libraries In the coming decade? 

Several issues appeared over and over again in the plans examined: increasing financial 
?' ' mp ! e T nUHon * UJMS ' to »«ess information resources beyond the library 
facility, the need of users for remote access to information, the gradual obsolescence of traditional 
library roles coupled with new opportunities provided by information technology, the trend towards 
^lu-ormatjon as commodity.- Working together to identify and discuss the issues facing academic 
heaUhjdences libraries can help individual libraries as they engage in their own strategic planning 

Recommendations: 

• Develop a planning approach, specifically aimed at academic health sciences libraries, which 
incorporates an understanding of important issues facing these libraries as a group, 

• Propose new and innovative service roles that may assist academic health science libraries 
better meet changing needs and services within their institution and community. 

5. What type of planning approaches appear to work best for academic health science libraries in 
various kinds of institutional settings? 

The planning documents clearly indicate that different assumptions about the type of planning 
appropriate for that particular library (and institution?) were.in operation. Indeed, it may well be 
that certain cntena used in this review to assess a planning document were considered inappropriate or 
unnecessary at individual institutional settings. Moreover, different content and development 
strategies may be more effective in one setting than another. 

Recommendations: 

• Investigate possible models of institutional contexts that might be related to library planning 
approaches. ° 

• Develop some form of an "institutional assessment checklist" that might assist the libraries to 
better understand the institutional planning context prior to the library's development of a 
plan. r 



Conclusion 



Overall, the fanning documents examined suggest a wide disparity and understanding about 
planning and especially the evaluation process. There is significant disagreement among the documents 
as to what constitutes a plan. Moreover, there appears to be minimal attention given to the 
institutional context in which planning takes place. Another concern was the limited attention given to 
implementation, i.e., who specifically would do what tasks at what points in time to insure that the 
objectives were accomplished. One had the sense with some of the plans that it was a document that 
"had" to be written, so it was. 



24 

5 3 



McChire/Van House 



AAHSLD Study 



Draft, September 26, 1991 



By examining strategic planning approaches and specific planning documents, this report has 
attempted to provide an initial model of strategic planning for academic health sciences libraries. 
While there was little agreement among the plans examined, most had specific strengths which 
contributed to the overall effectiveness of the plan. The key issues described above are gaps identified 
in the group of plans as a whole. By addressing these issues, academic health sciences libraries can 
improve their approach to strategic planning, can respond more directly to their environments, and can 
more effectively pursue their missions. 

The development of a planning, role-setting and evaluation manual for academic health sciences 
libraries should address these issues and incorporate these recommendations. Specifically, the concept 
of service roles can help libraries plan more effectively by (1) encouraging libraries to learn what roles 
their users need them to fill, (2) encouraging selectivity and prioritization, (3) facilitating the 
identification of strategic issues by casting the library in a specific, proactive role, (4) facilitating 
implementation of new programs by narrowing the library's focus, and (5) facilitating evaluation by 
enabling libraries to develop clear criteria for success. 



NOTES 
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Canada (AAHSLD, 1991). 
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3. For a more extensive description of service roles, and examples of roles for public libraries, see 
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October 6, 1991 



PURPOSE OF THE REPORT 

As part of this study, a number of data collection activities were undertaken including .three 
focus groups with participants from a variety of academic medical library settings. Focus groups consist 
of 7-10 people who possess certain characteristics who provide data of a qualitative nature in a 
focussed discussion. Focus groups provide valuable insights into the range of perceptions participants 
have about particular topics. Specific details of the focus groups held in conduction with this project are 
presented in Table 1: Focus Group Summary Table. The views and opinions of respondents on a number of 
topics were solicited. These topics were: 

• needs assessment of academic health science library planning and data management, 

• possible performance measures which might facilitate decision-making, 

• possible service roles and/or strategic planning scenarios for academic health science 
libraries, 

• suggestions for the planning manual. 

This report presents a synthesized summary of the focus groups, requirements and needs which the study 
team identified through analysis of the data, and specific recommendations for continuing action, and 
for the AAHSLD database and the planning manual. 
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Table 1: Focus Group Summary Table 



date 

6/17/91 



location 



Hamilton, Ont. 



number of 
participants 



6/29/91 



Atlanta, Georgia 



9/17/91 



Berkeley, California 



summary data 



Canadian librarians, 3 library 
directors, 2 technical services 
librarians, 2 systems librarians, 
and 1 faculty member. Average 
years in academic health science 
libraries: 52. 

American librarians, 5 from the 
South, 1 from New Mexico, 1 from 
New York, 1 from Chicago. 4 
library directors, 1 collection 
development officer, 1 systems 
librarian, 1 access services 
librarian, and the executive 
director of MLA. Average years in 
academic health science libraries: 
13.88. 

California librarians, 3 library 
directors, 2 deputy directors. 
Average years in academic health 
science libraries: h.fc 



EXECUTIVE SUMMARY OF THE REPORT 

The summary and analysis of the focus group findings are as follows: 

Academic health science libraries are at a turning point in management There are strong 
pressures to become better planners including limited resources and pressure for accountability within 
the institution. These pressures demand that skillful planning become an integral part of library 
operations. There are a number of contextual issues which impact a library's ability to do planning 
Key issues raised were: 

• the level of commitment to planning (institutionally and within the library), 

• the organizational structure of the library, 

• the lack of appropriate performance measures and planning techniques. 

Efforts to facilitate the planning process in academic health science libraries must be developed in 
light of these contextual issues. 

Skillful planning efforts will involve several initiatives on the part of academic health 
science librarians. The principal initiatives that emerged from the focus groups were increased 
involvement in institutional activities and planning efforts, and the demonstration of the value of 
library services to institutional missions and goals. 



2 



McClure/Van House/Hert 



AAHSLD Study 



Draft, October 4, 1991 



Focus group participants identified a variety of prerequisites to the accomplishment of these 
initiatives. The key prerequisites identified were: 

• increased education in planning and management, 

• increased education in data collection and data analysis, 

• more sophisticated data management skills, 

• the development of better and more valid measures of the value of library service, in 
particular, performance measures. 

• the establishment of service roles or scenarios to facilitate the integration of the 
library into institutional activities. 

Based on these prerequisites the study team has formulated a number of areas for further action. These 

areas center on education and training, and on the development of performance measures, service 
scenarios, and planning process models. 

NEEDS ASSESSMENT OF ACADEMIC HEALTH SCIENCE LIBRARY PLANNING AND DATA 
MANAGEMENT 

The first topic of discussion at the focus groups was the state of planning and'data management 
activities at the institution. This needs assessment was critical for establishing the current state of 
affairs as well as suggesting a number of factors that will influence the direction of the study team's 
further work. It also laid the groundwork for the further, more exploratory discussions which occurred 
during the focus groups. 

Planning 

There is a dear need for improved planning processes in academic health science libraries. The 
focus group participants uniformly expressed an interest in improving planning activities in their 
library given the need to utilize limited resources wisely and to justify the library's benefits to the 
institution. Most participants reported that planning activity was limited, both within the library 
and within the larger organization. Some planning does appear to be occurring, with several 
participants reporting that they had developed a planning document, been asked to review 
institutional planning documents, or felt that the institution was beginning to prioritize goals (by 
establishing areas of excellence and funding those at a higher level). The California librarians 
mentioned space planning as being quite important in their institutions. Only one participant reported 
that she was fully integrated into the strategic planning occurring at her institution. In general, even 
those participants who did report that there was some planning activity within the library or the 
organization still suggested that these two spheres of activity were not well linked or integrated. 

When asked specifically about what factors and conditions affected the library* s ability to do better 
planning and performance assessment, participants mentioned the following: 

• a leadership crisis in medical center administration, . 

• level of commitment to planning, 
•' organizational structure, 

• the continuing reliance on extensiveness measures and anecdotal evidence as primary 
tools for planning decisions at the institutional level, 

• the demand for certain types of data imposed by existing organizational commitments 
combined with limited time and resources to collect new types of data, 
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• the lack of techniques and measures to describe and justify the library's value in 
particular institutional activities, 

• the lack of planning and data management skills, 

• the lack of library staff interest and involvement in planning activities within the 
library and within the organization, 

• the conflict between the traditional perception of the library as overhead and the new 
need for accountability within the institution, 

• the lack of resources to implement what is planned. 

^foSTgTup^ 11 ^ 5 * faCt0rS relationshi P to P^ng activities constituted a major portion of 

Several environmental and organizational factors were reported as affecting the library's 
ability to do planning. Several participants suggested that a crisis in medical center leadership and 
the nigh turnover in medical center administrators led to a situation where the library was unsure 
about what goals to be pursuing. Medical center missions, philosophies and goals changed frequently as 
each new administrator pursued his or her own vision. One respondent suggested that the problem was 
exacerbated by the fact that medical center administrators were not trained in management techniques 
dunng the course of their medical school degree. 

Participants in the focus groups also reported various levels of commitment to planning, both 
within the library and at the institutional level. One respondent said that staff in her library were 
interested in a common vision but also wanted to go their own way. Another person said that at her 
institution it was "management by crisis." Another participant reported that planning at her 
institution was not designed around building a general vision of the future but was outcome oriented 
(that is, focussed on specific results). An institution's lack of commitment to planning might represent 
a belief that planning is not useful. Several participants suggested that strategic planning might not, 
in fact, . really be necessary and might just be a waste of resources. Another factor mentioned was the ' 
prevailing clinical attitude which causes people to deal with matters in a life and death fashion 
rather than in a long-term planning fashion. Planning efforts within the library are clearly affected 
by the prevailing attitude towards planning and the concomitant commitment of resources and time. 

Organizational structure also appears to play a role in a library's ability to do effective 
planning. Canadian libraries typically report to the University Librarian and find themselves trying 
to justify the cost and validity of their services to other librarians (who may feel slighted by the 
amount of resources expended on the medical library). Some of the American libraries also report to the 
University Librarian and a California librarian suggested that in his institution, department heads 
were the equivalent of "village chiefs" and that grand visions could not be imposed on them This led 
to short term solutions to problems and the belief that central planning slowed things down. 
Additionally, the medical library is slowed down because it can't do anything the other libraries on 
campus don't do. Other American libraries report to the dean of the medical school and compete with 
different institutional departments and may need to present different arguments. This theme of 
organizational structure as a determinant in planning capabilities surfaced in all of the focus groups. 

When asked what techniques they used to promote, justify, or sell the library to the 
administration, participants reported that they used interpersonal relationships, grassroots lobbying 
(getting other influential people to speak for them), anecdotal evidence, and traditional 
extensiveness measures (e.g. number of volumes) as well as demonstrating the ability to provide 
solutions. If necessary, these types of evidence were backed up with data from the Annual Statistics 
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volume. Comparisons were also made between their library and other peer libraries using the Annual 
Stasias volume. The peer group selected changed depending on what type of comparison needed to be 
made. Participants carefully selected who they would compare to in *hyV to made particular 
arguments to the adminisbation. A common approach was to high%.> Jn items in the Annual 
gfcftfrq voIum e *o visually demonstrate where the library was positioned relative to others. 
Concerns were raised that comparisons tend to get locked in even if they are no longer useful, and that it 
fa not always clear what libraries to compare to. One participant reported that his institution decided 
that it wanted to be in the top ten medical schools in the United States, but could not decide by what 
criteria to judge the top ten- 7 

WMetf^se techniques for justifying the library had some success, most participants were 
dissatisfied with their ability to justify the library and were looking for better measures and 
techniques. Specific types of measures that participants mentioned as lacking were measures of quality 
0f , i *™;' f^,^"^ n»nag«nent data (primarily costing figures). There seem to be several factors 
related to the lack of better techniques. One problem is the commitment to reporting traditional 
measures. The Canadian libraries appear to be in a particular bind in this respect Generally, the 
?*"»t A n J, raneS report t0 Universit y Librarian and therefore must report statistics to 
CARL/ARL as well as to AAHSLD. At this point, it appears that Canadian libraries would have to 
collect data and report new measures on top of the traditional measures already requested by 
CARL/ARL. Staff resources for such an undertaking are limited. The concern that new measures would 
be above and beyond what was currently being reported resulting in more time and resource expenditure 
was a concern for most participants. 

Along with the need to report traditional measures, the lack of new types of measures was also 
cited as a cause for ineffectiveness of planning. All participants eagerly awaited such measures. 
Several people reported that their administration was willing to listen to different types of data- 
administrators wanted to be told what types of measures would be more useful and why. During 
several of the focus groups, participants brainstormed to develop types of measures which might be 
useful. One of the most interesting developments in the sessions was the idea of linkages. Participants 
recognized that to justify the library's value to the institution in these times of limited resources 
required the demonstration of links between tibrary services and institutional missions. In essence, how 
did library services and activities contribute to fulfillment of institutional missions? These linkages 
are discussed more fully in the section entitled Performance Measures, 

Participants also recognized that the library and its staff also had a responsibility for 
improved planning activities. They felt that they and others in the hbrary did not have adequate 
training in management planning, or data handling. Skills such as data collection (particularly in 
terms of assuring validity and reliability of data), sampling, data manipulation (e.g. in spreadsheets 
or MIS), data analysis and environmental scanning were mentioned. The environmental scanning skill 
was recognized as being critical. One participant said that it was clear that staff on the front line 
have a lot of information about the institutional environment but it is still unclear how to collect this 
information, validate, and plug it into an MIS or into planning efforts. Another participant wished for 
a mood ring. While he was integrated into institutional planning efforts though service on committees 
he needed training in how to interpret what was going on (e.g. who the major players were) in ways to 
facilitate planning. ' J 

There is also an attitudinal component to planning efforts that needs to be addressed. Several 
participants said that library staff were not active enough in pursuing opportunities to be involved in 
planning. Another participant echoed this statement suggesting that library staff needs to be where 
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decisions are made, such as key committees, and must demonstrate credentials by being productive in 
these activities.. One person said that libraries frequently missed the chance to be involved in 
institutional planning because they were afraid to stick their necks out. Several people were hopeful 
that increased, strategic planning efforts could help create an environment in which people could get 
into a different mind set One person suggested that staff could be developed through coaching, 
training, and helping them to find opportunities to be involved. 

Associated with the perception of the lack of staff interest and involvement in planning is the 
conflict between the traditional perception of the library as overhead rather than a cost-generating 
center in the; institution. One person commented that if we wish to be more involved at the institutional 
level we need to move to a new management stance. This new stance would bring with it the 
responsibility to act in a different manner, including contributing financially to the institution and 
providing dear indication of the library's value to the organization. In the focus group where this was 
mentioned,, all participants felt that becoming a full partner in the organization was necessary. 

One final obstacle to better planning was brought out in the focus groups, the lack of time for 
planning and limited resources to implement what was planned. Planning efforts which go beyond the 
scope of what is possible leave people unwilling to participate further. 

Library Data Management Techniques 

There is a wide range of data management techniques and software in use in the libraries. Most 
of the participants felt that they had a handle on collecting the AAHSLD statistics but other data 
gathering was ad hoc and not well organized. Most of the Canadian participants had limited access to 
computerized tools. Most data in the Canadian libraries were still being collected manually, and were 
not l«ing incorporated into any computerized tool. The American libraries were much more fully 
computerized with all libraries having some type of spreadsheet or other application in use. One site 
was currently transferring data from the library's automated system to the data management software. 
None of the participants was currently using the machine-readable form of the AAHSLD statistics. 

POSSIBLE PERFORMANCE MEASURES 

Throughout the focus groups, dissatisfaction with current methods of demonstrating the value 
of library services to institutional administrators was expressed. Many participants reported a 
reliance on anecdotal data (which then might be backed up with evidence from the Annual Statistics ) 
and on the traditional extensiveness measures (e.g. volume counts). There seems to be several causes for 
this reliance, namely that librarians are not prepared to offer different types of data and that 
administrators seem to be unaware that other measures of the value of the library might be 
appropriate. 

This suggests a positive opportunity for libraries to develop new measures of performance 
providing that we can explain what they mean and how they are used. The clear sense from the focus 
group participants was that it was necessary for these measures to demonstrate the quality of services 
and what libraries do for people. Additionally the measures should be tied to particular institutional 
activities and missions; that it was important to establish areas in which the library's impact on 
institutional outcomes could be demonstrated. Focus group participants formulated a number of possible 
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institutional missions to which library services could be tied. Missions mentioned were: 

administrative planning activities, 
faculty programs both clinical and educational, 
research, 
patient care, 

medical school students' lifelong learning abilities, 
other professionals' lifelong learning habits, 
cost containment, 
curriculum change, 
curriculum quality, 
clinical librarianship programs, 

service to unaffiliated professionals and regional hospitals, 
development activities. 

Discussion of these missions led to the generation of some possible associated performance measures by 
participants. Measures suggested were: 



research grants that faculty have obtained through use of the library, 
demonstrating the use of the library in determining that one test was cheaper than 
another, 

• medical school residency placement due to students' information skills, 

• diversity and depth of programs. 

These measures represent just the tip of the iceberg in terms of measures which could be developed in 
relationship with particular institutional missions. 

POSSIBLE SERVICE ROLES 

During the course of the focus groups, participants were questioned as to possible service roles 
for academic health science librarians and libraries. A number of roles were mentioned. These were: 

involvement in lifelong learning, 
outreach to unaffiliated professionals, 
quality filtering or value-added services 

involvement in database choice, implementation, and use throughout the institution, 
medical informatics initiatives, 
AV services, 

computer services and other educational technology, 
I AIMS roles, 

curriculum development activities. 

Based on these suggestions, a set of draft service roles was developed between the second and third focus 
groups and presented to participants at focus group 3. Generally, they found the roles, as drafted, a bit 
unclear and made a number of comments for their improvement They also suggested that the roles were 
not detailed enough to facilitate planning activities. The study team is currently exploring new 
approaches to the development of service roles or service scenarios based on their comments. 
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NEEDS AND REQUIREMENTS 

Participants in the focus groups raised a number of issues concerning planning and management in their 
libraries. These are: 

1. Health science libraries must think strategically and planning is useful if it helps them to do 
that Planning must focus on key Issues of concern to the library and the institution. 

Librarians must be able to solve problems for people in their environment This means that 
they must be in the right place at the right time with the right information. This entails knowing 
what the right place is, what timing will work, and what information will solve the problem. If 
librarians cannot monitor their environments effectively they will not be able to do this. 

Planning must assist librarians in their monitoring efforts, helping them to understand their 
environments and think strategically about what to do next. Planning that takes librarians away from 
this effort is not helpful. ' 

Planning must be focussed on key issues of concern to the library and the institution. For many 
libraries, this may mean a focus on specific issues and outcomes (such as space utilization) rather than 
long-term strategic positioning. A focus on specific issues and outcomes may be very important in 
academic health science libraries where the culture emphasizes specific outcomes in specific time 
frames. Given this tendency, a planning manual should not be limited to techniques that are 
applicable only to strategic planning efforts. 

2. For medical libraries to survive in their current institutional contexts requires that the library 
play a more visible role in the life of the medical center. Assistance is needed in developing 
strategies and techniques for gaining and retaining participation in institutional planning and 
management efforts. There is a great need for techniques for reading the organizational 
climate. 

There was the general recognition that the library and its staff need to become full partners in 
institutional management activities if they which to remain a viable part of the organization. The 
participants suggested the need to demonstrate the value of the library to various institutional 
activities but perhaps more importantly emphasized the need to become active participants in 
planning and management at the organizational level. Library managers need to be, and see themselves 
as managers within the larger institution. 

While some strategies for becoming involved are being employed (such as committee work and 
high visibility), others need to be developed. Several factors were mentioned which have impacted 
library participation to date. These are outlined in the section on Requirements and needs for 
performance measures, service roles, and planning. These factors suggest that staff development 
activities relating to environmental scanning, planning, evaluation, and performance evaluation may 
be necessary. They may also suggest, that since every organization is different to some extent, a 
planning manual might be better focussed on strategies for developing and utilizing planning techniques 
than on lists of actual measures. 
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3. There is a crisis in medical school administration which adversely affects the library's ability 
to plan successfully. 3 

There is a crisis in medical school administration which adversely affects the library's ability 
to plan successfully. Focus group participants uniformly cited high turnover in the medical school 
administration, which has affected the library's ability to interpret the organizational climate and 
then plan based on that interpretation. Such a problem may argue for the development of planning and 
evaluation tools and techniques which function in the short run rather than the long run. 

Participants also suggested that medical school administrations had not identified for 
themselves, how to measure the school's performance. This makes it exceedingly difficult for the 
library to develop measures of effectiveness which are tied to institutional goals-Part of the crisis may 
be related to lack of adequate education and training of managerial activities. Some commented on the 
fact that neither they or their medical school manager counterparts had adequate managerial 
preparation (in a host of areas) to deal with the complexity of the issues that confronted them. 

4. Planning efforts at the library and institutional level are not well developed. Links between 
library service and Institutional goals need to be developed. 

While participants did cite some planning activity, both within the library and within the 
institution, it seemed scattered and still fairly undeveloped. There still appears to be a great reliance 
on anecdotal reports (which may be backed up with quantitative data). There was some feeling that 
institutional administrators did not know what to ask for and were waiting for instruction. The 
participants also addressed several problems they face in becoming integrated into planning efforts 
including lack of planning and data management skills as well as the lack of appropriate measures. 
Librarians frequently feel isolated from existing planning efforts in the institution for these reasons. 

, These discussions also suggest that a gap exists between the ways libraries have traditionally 

sold themselves to their institutions and the driving assumptions and forces behind current institutional 
planning (which influence the types of arguments and methods needed for justifying a library's value) 
Participants felt that new measures which linked to institutional missions would facilitate their 
ability to demonstrate the value of library services and provide useful tools for strategic planning at 
the institutional level. A number of possible linking areas were discussed in the section on Performance 
Measures. 

5. A number of new service roles were suggested for academic health science libraries. 

Participants also explored the need for new visions of library activities and services. It was 
felt that there might be a variety of ways to link to institutional missions which could be encapsulated 
into service roles which might serve as focussed visions incorporating a number of services and 
activities. Participants generated a number of these new visions or service roles. These were 
presented in the section entitled Service Roles. Several concerns were raised by participants about the 
development of service roles. The development of service roles should recognize that services are 
market driven in the health sciences environment and that they will change from year to year. They 
should also be non -prescriptive and allow for various levels of commitment and effort 

As the study team analyzed the focus group discussions related to service roles and drafted 
some example roles, it appeared that the roles were not detailed enough and did not capture contextual 
factors sufficiently to enable librarians to use them as tools to explore new directions in service. What 
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appeared to be needed were scenarios which are descriptions of how possible future states of being 
might evolve. Scenario development requires planners to make their assumptions about the future 
explicit and describe a future state of the organization in light of these assumptions and in light of 
possible organizational goals and resources. The study team is in the process of drafting example 
scenarios for exploration in later parts of the study. 

6. Data must be collected foe a purpose; data collection must be problem-centered. library data 
management techniques appear to be evolving and being recognized as an important 
management activity. 

Data collection efforts need to focus on particular problems to be resolved rather than on 
measuring processed. In considering what data to collect, librarians should first consider what their 
top problems are and then consider what data will help them in thinking about and solving these 
problems. This suggests that a planning manual should include information on how to prioritize 
problems and determine what types of data might be useful in resolving them. 

Most of the libraries were interested in or were already using some kind of data management 
technique. The level of data management techniques and the variety of application software currently 
employed, as well as the variety of needs seems to preclude the recommendation of particular software 
for use in data management tasks and suggests a more general focus in the planning manual. 

7. Different institutional contexts impact the library's planning and evaluation efforts. 

A number of contextual issues appear to impact the library's ability to plan. Issues raised in the 
focus groups include organizational structure, commitment to planning, organizational culture and the 
types of institutional missions to which the library must relate. These contextual factors and others not 
yet identified need to be taken into account as the study team develops models of the planning process. 
Figure 1 represents one approach to developing frameworks for planning within different organizations. 
The study team will continue to explore additional approaches for use in a planning manual. 



Figure 1 

LEVELS OF PLANNING COMMITMENT 

Institutional Perspective 

Library knowledgeable not know. know. not know, 

perspective and committed and comt'd not comt'd. not comt'd. 

knowledgeable 
and committed 

not know, and 
committed 

know, and not 
committed 

not know, and 
not committed 
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8. There Is a tension between the traditional role of the library as overhead and its new role as 
full partner In the organization. 

v i u_ undercurrent running through the sessions was the conflict between the institutional view of 
the library as "overhead" which receives funds for its support without contributing to the institution 
financially and the new role of library as full partner in the organization. Participating fully brines 
with it the responsibility to act in a different manner, contributing financially to the institution and 
providing clear indication of the library's value to the organization. In order to accomplish this goal, 
quantitative measures of the library's value need to be developed. 

9. Importance of education and training In the area of planning and evaluation of academic 
health science library services. 

Underlying much of the discu ssion was the concern that both horary managers and institutional 
managers are unprepared for the managerial responsibilities they have to fulfill. Thus, the 
development of a manual for academic health science libraries should be seen in the context of 
educating academic health science librarians as well as simply producing a manual. 

Moreover, the educational imperative includes an attitudinal component. The sense of the 
participants was that some academic health science library managers have some "doubts" as to the 
usefulness and/or appropriateness of performance measures; are the benefits from such planning and 
evaluation approaches likely to be worth the effort? As yet it is unclear to some that it is worth the 
effort. 

An implication of this issue is the importance of coordinating an educational program for 
AAHSLD members reg.trding MIS, planning, and evaluation as part of the overall project A theme 
within this effort is that librarians have to take on more of a "managerial" perspective; they need to 
recognize the importance and uses of planning and evaluation strategies. This educational effort can 
include the study team, but eventually, AAHSLD will need to take responsibility for an ongoing 
educational component-perhaps coordinating it with the MLA efforts. 

RECOMMENDATIONS 

Recommendations for Continuing Action on the Part of the Study Team 

Analysis of the focus group findings suggest several clear areas of action for the study team. 
These are: 

• performance measure development, 

• service scenario development, 

• education and training. 

• strategic planning process models 

Action on these fronts needs to occur in conjunction with the development of a planning manual for 
academic health science libraries. 

As the focus group results indicate, academic health science librarians are increasingly 
interested in planning and concomitantly in finding new ways and measures to justify their value to 
their institution. Better measures of the value of library services to particular institutional missions 



11 



65 



McClure/Van House/Hert 



AAHSLD Study 



Draft, October 4, 1991 



were discussed as useful tools for that justification process. Participants mentioned several possible 
measures and the study team needs to develop these further and supplement them with additional 
measures. This process is currently underway. Measures need to be tested both for their ease of use as 
well as for their utility. 

Service scenarios also need to be developed. The study team suggests that these will be more 
effect strategic planning tools than service roles based on discussions in the focus groups. Draft service 
scenarios have been developed based on the linkages to institutional missions discussed by focus group 
participants. These draft scenarios need to be tested to determine their utility for planning activities 
in a variety of institutional contexts. 

Comments from the focus groups also point to the need for the study team to consider the 
development of service scenarios and performance measures in the context of the medical center as a 
whole. In order to assess viable scenarios requires an understanding of how the medical center views 
itself. A focus group for medical center administrators might be a fruitful technique for assessing this 
group's views of management and planning. Additionally, the study team might look at models from 
other service environments and assess their relevance to the medical center context These other models 
might be used as the starting point for the development of service scenarios and performance measures 
to be used in the medical center environment 

The study team should also focus some of its energy on education and training activities. While 
participants in the focus groups were enthusiastic about using new measures and increasing planning 
efforts in their libraries, they had some concerns about their planning and data management skills and 
about their ability to develop new measures and services. The study team can play a role in the 
necessary educational activities both through the workshop to be held on November 9, 1991 and also 
through the incorporation of pertinent information in the planning manual. It should be recognized 
that the responsibility for ongoing and long-term educational activities cannot rest with the study 
team and alternative sources for educational support need to be identified. 

Academic health science libraries are at a turning i?oint in management. Factors such as 
limited resources and the increasing need for accountability within the institution necessitates that 
librarians become better managers of day-to-day activities as well as visionary strategic planners. In 
many cases, however, the requisite skills and attitudes are not fully developed. The study team is 
addressing these problems in a variety of ways one of which is the development of planning models 
which are effective for these libraries. 

Strategic planning process models which work far academic health science libraries need to be 
developed. Based on an analysis; of focus group finding:; , it seems clear that normative planning process 
models may not be effective given the institutional context and current environment of academic health 
science libraries. The study team is exploring new approaches to planning which will be examined for 
their utility in a variety of settings. These new approaches also need to be represented in the planning 
manual. r ° 

Recommendations for the AAHSLD Database 

As we discovered in our study of the usage o'/ the AAHSLD Annual Statistic volume and also in 
the focus groups, the statistics are widely used for comparison purposes. It is not as clear that they, as 
they currently exist can serve the other important functions which focus group participants mentioned 
such as justifying linkages and demonstrating the value of library services to the institutional 



12 

66 



McQure/Van House/Hert 



AAHSLD Study 



Draft, October 4, 1991 



adminstradoa Jt nay be that the statistics volume wiU continue to serve the comparison need, but that 
the manual to be developed in this project satisfy other planning and management needs. 

The volume and the database could be improved to facilitate the comparison process. One clear 
need is a performance measure section. Comparison of different institutions in a performance measure 
environment will improve the library's ability to Justify itself to management. A list of possible 
«r*asures that can be generated from currently existing data has been developed is being reviewed to 
determine which measures are of value. 

_ .. A ^ seco " d P Iace where improvement might take place is in the comparative groupings. Our 
findings from the study showed that there was a great amount of dissatisfaction with the groupings. 
ThepUnning and organizational contexts which are established for the planning manual need to be 
integrated into the development of the groupings which are used in the Annual Statistic* volume. 

Finally, there needs to be increased marketing of the machine readable form of the database 
That machine readable form needs to be as generic as possible so that it can be used with a variety of 
software packages and MIS's. While some comparisons are possible with the paper volume, others are 
not, and as focus group participants mentioned, the institutions to which they wish to compare change 
for different types of decisions. Only with the ability to reformulate the data as necessarycan this 
comparison process be facilitated. Instruction on how to manipulate the data needs to be provided as 
well. r 



Recommendations for the Planning Manual 

During the focus groups, participants were asked for their suggestions for the planning manual 
and they provided a variety of insights. Along with the suggestions that the manual be easy to read 
and clearly written, the following suggestions were made by participants. 

• Provide information about how to load and use the machine readable AAHSLD 
statistics. 

• Provide information on how to use data in decision-making tasks. 

• Focus on evaluation techniques which are not to labor-intensive given the amount of 
time already spend collecting such data. 

• Do not advocate particular software products since each library will have different 
needs. Try to make the approach as generic as possible. 

Provide assistance on how to explain the use of performance measures to library and 
institutional administrators. 

• Consider planning techniques that will work in the short-run (since the institutions are 
changing so rapidly). 

• Provide information on ways to show the difference the library makes to the 
organization. 

• Give assistance in the development of service scenarios, 

• Provide information on how to cost things out 

• Include methods on relating data with examples rather than just providing a list of 
performance measures. 

These specific suggestions from participants as well as the other discussions which occurred during focus 
groups lead to a variety of suggestions for the planning manual. 
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The key points made by participants regarding the planning manual focus on a need to provide 
a "pick-and-choose" type of environment where users of the manual are free to choose different 
institutional contexts, different library roles or links which need to be maintained, or different levels of 
commitment to planning (both within the library and within the institution) in order to assess which 
planning strategies, techniques, and measures are appropriate. Possible elements of the contexts raised 
by participants are: links to the medical center mission, different library roles, different 
organizational structures, and different levels of commitment Clearly, academic health science 
libraries in different situations may require different planning and evaluation strategies. 

Focus group discussions also indicated the need for the development of useful performance 
measures and their incorporation into the manual. Along with presenting particular measures, the 
"pick-and-choose" environment being considered for the manual suggests that information about the 
development of new measures should be included and that measures be tied to particular institutional 
contexts if appropriate. 

The study team is also considering the incorporation of service scenarios in the manual as tools 
for focussing vision and planning efforts. One of the most striking results from the focus groups was the 
need for new visions for service and action within the library in order to respond to changing 
institutional environments. Frequently new missions and services are difficult to develop without 
examples from which to work Service scenarios start with detailed descriptions of "what might be." 
Planners compare and assess these possible visions for the future in light of environmental conditions in 
order to generate goal statements which are then used to develop objectives and plans for action. The 
benefit of scenarios are that concrete examples are provided to stimulate the creative process of 
visionary mission and goal setting. Again, as with the case of the performance measures, focus group 
results suggest that a menu of scenarios be provided to allow libraries in different organizational 
contexts to utilize tto scenarios that would be most useful in their particular planning efforts. 



• THE NEED FOR STRATEGIC THINKING 

Throughout the focus group process it was clear that there is a need for AAHSLD members to 
"think strategically" about the role of academic health science libraries in the future. Strategic 
thinking requires: 

• Having a vision 

• Making choices regarding what will and will not be done 

• Leveraging resources 

• Targeting services and resources to specific deintele 

• Exploiting competitive advantages' 

• Positioning the library in the institution and in the community. 

A strategic posture is opportunistic; it is one that attempts to set agendas rather than responds to them; 
and it is one that recognizes the importance and use of power and politics. Strategic thinking must also 
be combined with leadership, vision, and bold decision making. Any approach for the development of 
a planning manual, performance measures, service scenarios, and data management should include a 
, strategic planning perspective. 
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EXECUTIVE SUMMARY 

The AAHSLD data system must provide data for the internal comparison and measurement needs of 
library management as well as for the institutional management needs of the health sciences center 
administrator. Library managers are concerned with efficiency and effectiveness of their operations. 
Health science center administrators are concerned with the library's contribution to outcomes in the 
areas of teaching, research, patient care, and with overall cost management and containment 

Academic health sciences libraries and their medical centers use AAHSLD data to compare their 
library with other libraries in similar organizations. The database must be designed to allow libraries 
to identify their own comparison organizations and manipulate the data accordingly. The data must be 
comparable across institutions and over time. The measures and data collection methods must be 
uniform. 

The choice of measures to be produced from the AAHSLD database must be based on three major 
criteria: the quality of the data; the value of the data for decision-making; and the level of effort 
required to collect data. Several, possibly contradictory, needs for the measures and data elements to be 
included have been identified. The data elements must be fairly stable, and varied enough to 
accommodate a wide variety of needs. The measures included must keep up with changes in information 
technology, in the health sciences, in the academic health sciences center environment, and with 
academic health science library functions. However, the level of effort required to collect the data 
must be minimal Data collection methods must be easy and explicitly defined. 

It is possible for the study team to identify many potentially useful measures. We consider it more 
important, however, to determine the current questions and problems for which data are needed and the 
major strategic directions, roles, or functions in which AAHSLD libraries will be engaged and for which 
information will be needed in the future. Current questions and problems already identified by the 
study team for which data are needed are: 

• The impacts of delivery of information to remote sites 

• Information technology use 

• Costing of services 

• Information on income generation activities 

Strategic directions, roles, and functions also need to be considered in measurement development: 

• The development and implementation of new visions for the library (see report "Example 
Vision Statements for AAHSLD Libraries") 

• The increasing need for accountability within the institution 

• The need for the library to respond proactively to changes in information technology and 
within the institutional context. 

The study team is continuing to work on identifying and specifying these directions and roles. 
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The next phase of this study, in its treatment of measurement issues and the AAHSLD statistical 
system, should focus on: 

• Continuing identification of current and enduring information needs, as well as 
identification of important future health sciences library functions, roles, and strategic 
directions, and THEN specification of the data needed to facilitate associated management 



• The assessment of methods used to collect the current data and the development of easy 
methods for collecting high-quality data 

• The linkages needed between library goals/services and the statistical system to evaluate the 
success with which services are accomplished 

• The design of the data base and MIS system to provide those data in a flexible system that 
would allow libraries to construct their own data sets and analyses to meet local needs 

• The development of the AAHSLD data base and individual library management information 
systems that support strategic planning. 

In short, the development of the AAHSLD data base raises a range of complex issues and concerns. 
Figure 1 represents a sampling of some of the issues that will have to be addressed by the study team as 
it investigates the development of the data base. 



MEASUREMENT AND EVALUATION IN AAHSLD LIBRARIES 

Participants in the focus groups held in conjunction with the project were asked about current 
measurement activities in their libraries. All participants were collecting data for AAHSLD and the 
Canadian libraries were collecting CARL/ARL data in addition. Data from the AAHSLD Annual 
Statistics were frequently used to make comparisons between the library and other similar libraries. 
All participants pointed out that comparison groups changed based on the argument that the library 
was trying to present to the administration. 

Other measurement activity was limited primarily because of lack of time and resources. Given 
existing commitments, any new types of data needed for decision-making would need to be collected in 
addition to what is already being collected. Participants were dissatisfied with this. They felt that 
the increasing need for accountability to the institution was creating the need for new measures 
(including performance measures) and techniques for demonstrating their value to the organization. 

During discussions of ways to demonstrate the library's value to the institution, participants 
explored the possibility of demonstrating the library's contribution to particular institutional missions. 
Linkages suggested were linkages to administrative planning activities, faculty programs both clinical 
and educational, research, patient care, medical school students' lifelong learning habits, cost 
containment, curriculum change, clinical libra rianship programs, service to unaffiliated professionals 
and regional hospitals, and development activities. Demonstration of these types of linkages will 
require that new measures be developed and the data associated with them be collected. 
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PURPOSE OF THE REPORT 

The purpose of this part of the study was to assess AAHSLD library requirements for performance 
measures and the uses of the AAHSLD database in producing such measures. This report addresses: 

• The current and possible use of measurement in AAHSLD libraries and the role of the AAHSLD 
data base in measurement activities. 

• The study team's recommendations for the performance measures that should be produced from 
the database and for the data elements that should be contained in the database. 

• The study team's assessment of how these measurement efforts could be linked to the strategic 
planning process. 

• The implications of these findings concerning measurement for the design of a library 
Management Information System. 

Major sources of information for this report were the three focus groups that met with the investigators 
(the discussions of which are summarized in another report, "Summary and Analysis of Academic 
Health Science Library Focus Groups*'). In preparing this report, we also reviewed the AAHSLD 
Annual Statistics and the Hert and McClure report, A Study of the Use of the AAHSLD Annual 
Statistics in Member Libraries , as well as the literature review done as part of this project. We also 
drew on our extensive experience with measurement issues and approaches in libraries. 



Figure 1 

Factors Affecting the Design of the AAHSLD Database 
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Purposes of Measurement 

Measurement in AAHSLD libraries is used for two major purposes: 

• Within the library, for making decisions about resource allocation and program design. 

• Outside the library, to communicate with people, primarily the pe rent organization in order to 
demonstrate the library's value and to justify its budget 

two uses place very differed Library managers 1 

mam concern is internal comparisons. They use data to determine whether the horary should seek to 
accompl sh a particular goal through one program of activity or another; or how the library should 
aUocate ts resources among several goal or program areas. Additionally, since library managers are 
detail m hhnTy act,vities ' m «y *« fen«»« with its operations at a high level of 

Health sciences center administrators also make use of library data. They are concerned with the 
efficiency and effectiveness of the entire organization. They may be unfamiliar with the particulars of 
die library but have to make decisions about the library's programs and resources. They have to 
decide what proportion of their budget to allocate to the library versus other program areas. Their 
primary concern is with the library's contribution to the health sciences center's overall operations. 

The importance of different uses for measurement should not be underestimated. Different data 
may be needed; different language is certainly needed; different methods of data presentation are 
generally needed, as well. Yet academic health sciences libraries, like most other organizations, 
generally rely on a range of data for both internal and external purposes. The AAHSLD Annual 
SjaJisfca volume is likely to be only one of many sources that might be used. --ua-i 

Measurement and Evaluation 

Measurement does not in itself assess how well the organization is performing. Evaluation is the 
process of comparing measurement data to some standard of performance or expected level of 
accomplishment. 

Findings from the focus groups make it clear that library managers are increasingly concerned with 
assessing and demonstrating the library's contribution to its parent organization's mission. The 
effective library contributes to the medical center's teaching, research, and patient care, as well as 
contributes to the effective management of the institution. This kind of a^ment requires an 
evaluation of the health sciences library's contribution to th e medical center's outcome. 

The area of outcome assessment is of growing importance in medicine, but is fraught with problems 
whichdenve from our inadequate understanding of and ability to assess causality. Although there 
have been some attempts to determine libraries' contributions in such areas*, the role of library services 
in producing medical center outcomes is generally indirect Library services contribute to research, 
teaching and clinical practice, but may be just one component of many. Moreover, the availability of 
library resources does not ensure that the library will be used, or used appropriately. 



5 



73 



Van House/McClure/Hert 



AAHSLD Study 



Draft, October 23, 1991 



As a practical nutter, in the research-oriented environment of the academic health sciences center, 
assessment of the impact of library materials and services on health sciences center outcomes will itself 
require research. At the moment, it is unclear which outcome measures are (1) generally useful for 
AAHSLD libraries, and (2) especially useful for individual AAHSLD libraries. Research will require 
work to explore high quality time-series data elements currently not available in the AAHSLD data 
base. 

In future phases of this study, therefore, it would be appropriate for the study team to review the 
current state of outcomes research and dejgmjne, what H" ds of research would be needed to assess the 
impact of library resources and services on AAHSLD lib r ary outcomes. The actual empirical research 
on library contributions to health sciences center outcomes will require considerable time and resources. 



Making Comparisons 

* 

In place of such a definitive assessment of library performance, academic health sciences libraries 
and their medical centers rely heavily on comparisons with other, similar organizations. The library 
staff compare the library to other similar ones to determine how well they are doing, and whether 
improvement is necessary and possible. Health sciences center administrators compare their library's 
resources and services with those of comparable institutions to determine the quality of their library 
and the adequacy of its resources. In the Hert-McClure study of the AAHSLD Annual Statistics. 
"providing comparisons between your library and others" was the most frequently-cited use of the data. 

A critical question, then, is which organizations are used as the basis for comparison. We have 
learned in our interviews that each AAHSLD library and its parent institution determines its own 
reference group. These may be neighboring institutions; medical schools competing for the same pool of 
students; research centers competing for research funds; or other campuses of the same university; and 
the like. Furthermore, the comparison groups may change depending on the issue . A medical school 
may compare itself on one issue with those with whom it competes for students, and on another issue 
with those with whom it competes for research grants. The AAHSLD data base need not - cannot - 
define groups of comparable organizations, or characteristics by which organizations could be grouped. 
Instead, the database must be de signed to allow libraries to identify their comparison organizations 
and manipul ate the data accordingly. 

This pervasive use of the AAHSLD statistics for comparisons has major implications for the 
development of the AAHSLD data base. When data are used for internal purposes, there can be local 
variations in data collection to accommodate local needs. If, however, the data are to be compared 
across institutions, as the AAHSLD data are, then strict uniformity in data collection is necessary 
regardless of local needs. 

For example, one library may want to count questions on how to use its online catalog as reference 
questions, because the catalog is complex and the questions are time-consuming and require highly- 
skilled staff. Another library with an easier catalog may not consider catalog questions to be reference 
questions. If the data are for internal purposes, the staff within one library must all count catalog 
questions the same way, but the libraries may differ from one another. If the data are for comparisons, 
however, the libraries must define reference questions according to some agreed-upon criteria such as 
complexity (all questions requiring professional skills are reference questions), or topic (questions about 
how to use the catalog are not reference questions); and then all the libraries must count consistently. 
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Two major requirements for the AAHSLD database and annual statistics report, then are: 

• The data must be comparable across institutions and over time. The measures and data 
collection methods must be uniform, 

• The database must be easily manipulated so that each library can define its own comparison 
group as needed. Predefined comparison groups are of limited value. 

An implication of these requirements is the re-thinking of the type of data needed to make useful 
comparisons between the library and other sets of libraries on specific types of data and outcomes. 

CRITERIA OF MEASURES FOR THE AAHSLD DATABASE 

The choice of measures to be included in the AAHSLD database must be based on the followine 
criteria: 5 

• The quality of the data 

• The value of the data for decision-making 

• The level of effort required to collect them 

Complicating the development of the measures is the consideration that the knowledge, skills, and 
uses of the measures are likely to vary considerably across the various AAHSLD libraries. 

Data Quality 

Good measures possess the following characteristics: 

• Validity: valid measures measure what they are intended to measure. Circulation, for 
example, is not a valid measure of collection use in a situation where much use is in-house. 

• Reliability: reliable data are data collected in the same way at different times and across 
different institutions, so that if there are differences in the data, they reflect differences in 
that which is being measured rather than differences in how the measurement was done. 
Reliability is easier to achieve with some measures than with others: the measurement of 
reference transactions, for example, requires that a consistent definition of "reference 
transaction" be applied by a large number of staff repeatedly. This is more difficult to do 
reliably than to program a circulation system to count circulation transactions. 

• Usefulness: the number of possible performance measures is unlimited. Good measures are those 
that are useful, that is, that meet a particular need or purpose. 

• Understandabilitv: for measurement data to be used, they must be understandable to those who 
need to use them. As we already noted, this may be internal or external decision-makers, with 
more or less extensive knowledge of the library. 
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Data must have all these characteristics if they are to exhibit the highest quality. The better the 
quality of the data, the more value of the data, and the increased level of effort in collecting the data. 



Value of the Data 

Data are used for decision-making. Which data are most useful depend on the uses to which they 
will be put That is; the strategic priorities, the current problems, and the decisions confronting the 
users of the data determine which data should be collected and used. This may seem to be a tautology, 
but it is important for this project that we note that it is difficult to define a limited set of most-useful 
measures because as circumstances and priorities change, the libraries' data needs chang e. 

Differences in the health sciences environment, the medical center environment, information 
technology, and the libraries will result in different problems and therefore different data needs. All 
these areas are rapidly changing. The fact that the data are used both within the library and within 
the medical centers makes even more variation is possible. 



Level of Effort In Data Collection 

Data collection and analysis are costly in staff time. Careful decisions must be made about what 
data to collect and how. An added complication to AAHSLD data collection emerged from our 
interviews. When libraries collect data about their own operations, they choose the level of effort and 
quality of data according to their own priorities and needs. But when libraries report data to 
AAHSLD. thev are collecting data for other libraries, not for themselves. They have little incentive 
to expend much time and effort Internal needs will always take precedence. 

The implication for the AAHSLD data collection effort is that, taken as a group, the libraries will 
never devote substantial time and ef fort to AAHSLD data collection. Any measure that requires 
substantial data collection is therefore immediately suspect, because it is unlikely that many libraries 
will have devoted much effort to collecting reliable and valid data. 

Concerning the choice of measures to be reported, then, several, possibly contradictory, needs are 
apparent 

• The measures reported in the AAHSLD database must be fairly stable. Libraries need to be 
able to investigate trends and changes over time, so the same data must be included year after 
year. The effort required to collect the data in so many libraries requires that definitions and 
procedures change fairly slowly to allow for maximum coordination. 

• The number of measures reported must be large and varied enough to accommodate a wide 
variety of data needs. 

However 

• The level of effort required to collect the data must be minimal. 
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• As much as possible, the data collected must be those that the reporting libraries themselves 
will wish to use, both internally and externally. 

• Data collection methods must be easy and explicit; the more direction and help that libraries 
get in collecting data, the more likely they are to do so in the way suggested by AAHSLD. 

• The measures included must keep up with changes in technology, in the health sciences, and in 
the health sciences center environment, and with health science library functions. 

• Libraries need to be able to generate their own. measures and comparisons from the data 
elements reported in the database 

These are important requirements that must be kept in mind in the development of academic health 
science library measures. 



PERFORMANCE MEASURES FOR AAHSLD LIBRARIES 

The term "performance measures" encompasses a range of types of measures that address 
organizational performance: 

• Inputs, or resources 

• Processes by which organizations convert inputs to outputs 

• Outputs, (e.g., services delivered) 

• Outcomes, or the effect of library services. (e.g.; the results of information use). 

In libraries, as in other kinds of organizations, especially service organizations, the tendency has been 
to collect data on inputs because they are easy to collect, unambiguous, and easily compared. Outputs 
and outcomes, on the other hand, are none of these. So although organizations are generally more 
interested in outputs and outcomes, they have tended to collect data on inputs. There is an increasing 
need for AAHSLD libraries to be accountable to their institution and present data that provides 
evidence of the library's value to institutional outcomes. This need demands that the AAHSLD 
database be reexamined for its ability to provide data on outputs and outcomes. 



Choosing Performance Measures for the AAHSLD Data Base 

The AAHSLD data base consists of a set of data elements for which raw data and summary 
statistics can be reported. Current data elements included describe clientele; inputs or resources, 
including library materials, staff, and hours open; expenditures; and use measures. Additional 
measures of the following types can be constructed from the available data elements: 

• Changes in inputs and/or outputs 

• Unit output per unit input (e.g. reference transactions per FTE librarian) 
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• Output per unit of time (e.g. circulation per hour open) 

• Output per member of primary clientele (e.g. circulation per student) 

• Ratio of one type of output to another (e.g. ratio of total information contacts to gate count) 

• Ratio of one type of input to another (e.g. net volumes added per librarian FTE) 

• Subdivisions of inputs (e.g. holdings by type of material or classification area) 

• Cumulation of outputs (e.g. external circulation + reserve circulation + in-house use ■ total use) 

• Subdividing various outputs (e.g. external circulation for different parts of the collection) 
Some other measures could be added using new data collection, including: 

• Availability of various resources, e.g.the probability that a resource (library material, study 
space, etc.) will be available when user wants it 

. • Success rate: proportion of users or uses that are successful 

Numerous added analyses of the financial and expenditure data are possible: 

• Expenditure per unit output 

• Expenditure per unit input 

• Expenditures per member of primary clientele 

However; development of performance measure out of context of information needs, library services and 
goals, and strategic directions for the future of the library is inappropriate. A clearer sense of these 
issues is needed as a prerequisite for the development of measures, the design of the AAHSLD data 
base, and the development of library management information systems. 

Recommendations for the Data Base 

The AAHSLD Annual Statistics Editorial Board has, over the years, give considerable thought to 
the data to be collected and reported. The most useful approach of this study is not to second-guess this 
committee, or to suggest additional measures (of which, as the list above shows, there are many), but to 
think through some of the more fundamental issues about what is in the data base and how the data 
are presented. - 

The discussion above concludes that the data to be included in the database and report should be 
determined bv the audi ence's information needs. Rather than detail possible measures, we find it more 
useful to consider 

• What are the current questions and problems for which data are needed, and 
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• What are the major strategic directions, roles, or functions in which AAHSLD libraries will be 
engaged and for which information will be needed? 

The second question is being addressed by the part of this study that is concerned with the 
development of strategic roles and visions. 

For the first question, we have identified from the focus groups some areas for which measurement 
is needed. These are described below, with some possible measures. More development is needed in 
each of these areas. In the next phase of this study, we will work with AAHSLD librarians and the 
statistics editorial committee to (1) identify areas in which measurement is needed and (2) develop 
specific, do-able, useful measures. Moreover, it must be stressed that the identification of problems for 
which data are needed is closely intertwined with the identification of health sciences library roles 
and strategic directions. 



AREAS WHERE PERFORMANCE MEASUREMENT IS NEEDED 

L Delivery of information to users at remote sites. 

Medical centers are increasingly decentralized, with library users physically scattered. Advances 
in information technology allow users to access the library's databases and request documents without 
ever visiting the library. Overall, this is a positive development, reducing the time required for users 
to meet their library-related needs and making the library accessible at all hours. Remote users, 
however, are less likely to get assistance from the library staff, and cannot as easily browse the 
library's collection. The challenge for health sciences librarians is to provide high quality and rapid 
service to remote users. 

Some related data needs are: 

• Numbers of remote uses of various types (catalog searches, other database searches, document 
requests, documents delivered) 

• Speed of service to remote users, including speed of delivery of information 

• Availability of service to remote users (e.g. proportion of time that a port is free for remote 
searching). 



2. New information technology 

As new technology is adopted by libraries, the AAHSLD statistics on resources and services must 
include them. The most obvious are CD-ROM workstations and databases and end-user searchable 
online databases. 

Possible measures are: 

• Number of CD-ROM workstations 

• Number of databases available on CD-ROMs 
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• Number of CD-ROM searches performed (requires that the off-the-shelf software has been 
designed to count mem; or a log for user self-reports) 

• Number of online databases available for end-user searching 

• Number of end-user online database searches performed. 

An interesting issue mat requires further thought is the role of the library in assisting users in end- 
user searching. With services like Lonesome Doc and Grateful Med, the library is no longer an 
intermediary. The library does, however, provide support to the medical professional doing his/her 
own searching. The medical center is likely to want information about the extent of end-user searching 
by its members regardless of the library's role 



3. Income-generation 

Some health sciences libraries are under pressures to generate income for their parent institution. 
They may do this by contracting with other organizations for services; by getting grants; or by producing 
and marketing products (such as databases and software). Such activity would result in the need for 
data on: 

• Income relative to expenditures 

• Income and expenditures per unit of service or output 

• Various financial measures related to costs and income. 



4 Outcomes measures 

As health sciences libraries are increasingly pressed to justify their budgets, librarians and their 
superiors increasingly want data on the library's outcomes: how do people use the information that 
they get from the library? What difference does the library make in the organization's research, 
teaching, and patient care? To what degree does the library contribute to the success of the host 
institution? This is a very difficult area but one that needs more investigation. 

The most common method of assessing library outcomes is to query library users, by way of a survey 
or focus groups, about the library's contribution to their activities. Surveys and focus groups are very 
time-consuming. The quality of the results is limited by respondents' (1) willingness to participate and 
by their ability to (2) remember and (3) evaluate the library's contribution to their work. Such 
methods are highly imperfect. However, this is an area that is critical and further investigation is 
required into possible methods of assessing the library's contribution to research, education, patient 
care, and administrative activities. 
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5. Costs 

Accountability requires information not just on outputs and outcomes, but on costs. What resources 
are used to produce the outputs required? Many health sciences centers are acutely concerned with costs 
and income, and have begun to place unprecedented demands for cost information on their libraries. 

In making cost comparisons across libraries, resources have to be charged similarly. The uninformed 
comparison of costs across libraries is extremely hazardous. Costing of library operations is very 
difficult especially when the library is embedded within a larger organization, as is usually the case. 
Accounting conventions differ. More confusing, however, is the variation in how resources are charged 
to » unit. Such bask issues as whether and how the library is assessed for its space and utilities make 
cross-library comparisons very sticky indeed. 

Withiiv the library, the attribution of costs to cost centers is also extremely difficult For example, 
if a journal is used for in-house consultation and photocopying, for out-of-library circulation, and for 
copying for interlibrary loan requests, how is the subscription cost to be divided among those uses? 



MANAGEMENT INFORMATION SYSTEMS AND THE DATA BASE 

The ability of the library to organize, manipulate, and use some form of a management information 
systems (MIS) for decision making is a key issue yet to be faced by AAHSLD libraries. Collecting the 
appropriate data and developing appropriate measures is necessary but not sufficient for effective use 
of the AAHSLD data base. Each AAHSLD library must also be able to organize and manipulate the 
data that is (1) submitted to the AAHSLD data base and (2) collected locally and specifically to assess 
the library's services and justify a range of library activities. Currently, AAHSLD libraries do not 
have meaningful systems of data management within their libraries. 

An effective machine readable MIS provides library decision makers with accurate and timely 
information that improves the basis by which decisions are made. The focus of the MIS is to capture, 
organize, analyze, compare, and report a range of data describing library activities, services, and 
management The MIS should support a number of different types of decision making: 

• Operational decisions (day-to-day decision making) 

• Strategic planning decisions (determining appropriate objectives and accomplishing those 
objectives) 

• "What if* questions (if we increase expenditures on a particular service, what would the 
impact be on another service?) 

• Exceptions (in-house use of periodicals has decreased drastically during the last month-why?) 

• Resource control (ensuring that resources are expended on appropriate activities or items). 

An effective MIS for AAHSLD libraries would draw upon data submitted to the AAHSLD data base 
but would also include a range of data elements needed for that particular library as well. Performance 
measures developed for the AAHSLD data base may be augmented with performance measures used in 
an individual AAHSLD library MIS. 
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Typically, AAHSLD libraries have a broad range of management data resulting from both internal 
and external data collection activities. Indeed many different individuals throughout the library may 
have or produce such data. But not everyone may know what data are available, the data may not be 
reliable, and Utile thought may be given to determining which data to collect and how often. Further, 
the data may not be made readily available to the decision makers who need it the most, nor might the 
data be presented in a format that is understandable by the library decision maker. Most AAHSLD 
libraries are unable to: 

• Comprehensively capture, organize, analyze, and report information that describes the 
library's environment and library-related activities and services 

• Identify clientele user information needs systematically and analyze those needs both for day- 
to-day decision making as well as for long range planning. 

• Demonstrate the overall effectiveness and efficiency of specific library services and activities 

• Draw upon the AAHSLD data base for data that can be manipulated and analyzed in light of 
local information needs 

• Simplify and standardize the library data gathering and reporting activities to ensure the 
collection of quality data for use in library decision making 

• Incorporate data from the AAHSLD data base or from a local MIS into th* strategic planning 
process. 

AAHSLD libraries need to develop a locally-based MIS that draws upon the AAHSLD data base, 
utilizes additional local data as appropriate, and produces outcomes-oriented information and 
measurement of use for strategic planning. 



THE STRATEGIC PLANNING CONTEXT 

Development of the AAHSLD data base, design of performance measures, and the need for 
management information systems must be considered in the larger context of strategic planning (»x 
report "Review of Academic Health Science Library Planning Documents"). At its most general levul, 
strategic planning simply means: 2 

A disciplined effort to produce fundamental decisions and actions that shape and guide what the 
organization is, what it does, and why it does it At its best, strategic planning requires broad-sca'e 
information gathering, an exploration of alternatives, and an emphasis on the future implications 
of present decisions. 

The importance of a future perspective in strategic planning is essential. It is for this reason that the 
study team has spent some time on developing possible vision statements (see report "Example Vision 
Statements for AAHSLD Libraries"). 
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Yet, a future perspective for successful strategic planning must be grounded in the ability to know, on 
an ongoing basis, how well the library currently does what it says it is doing. Knowing how well the 
library does what it says it is doing requires the library to: 

• Have goals and to be able to articular what it should be doing and what its vision of itself is for 
the next 3-5 years. 

• Have a systematic means of identifying, collecting, organizing, and analyzing a range of 
environmental and library data. 

• Have, measures that assess the performance of the library for key service activities and to 
determine the degree to which the library actualized the vision it has articulated. 

Measurement, evaluation , and data management should not occur in a vacuum. They occur in the larger 
context of strategic planning. Thus, a key requirement that must be addressed in later phases of the 
project is integrating the measurement, data collection/management, and vision development processes 
into coherent strategic planning for AAHSLD libraries. 

Figure 2 represents an evolving strategic planning context for AAHSLD libraries. The figure 
suggests that a key ingredient is to first develop a vision statement of appropriate library services and 
activities. Next there is an assessment of current library services/activities. Strategic planning then 
develops a process for moving the library from existing services/activities to those identified in the 
vision statement. The AAHSLD data base as well as the library's MIS support the strategic planning 
process. 



SUMMARY AND RECOMMENDATIONS 

Measurement is used for two major purposes: (1) internal decision-making and strategic planning 
regarding resource allocation and program design, and (2) communication with people outside of the 
library, primarily the host institution, to demonstrate the library's value and justify its budget. The 
AAHSLD data must address the concerns of both library management and the health sciences center 
administrator. The next phase of this study will more thoroughly assess those concerns. But in this 
phase the study team has learned some useful things about the AAHSLD database and its uses. 

• AAHSLD library managers use the AAHSLD Annual Statistic* primarily for comparative 
data with selected other libraries to address current concerns. These concerns vary, and the 
reference groups used vary. 

• Health science center administrators are concerned with the library's contribution to outcomes 
in the areas of teaching, research, and patient care; and with costs. 

• Existing data in the AAHSLD data base is not seen to provide linkages between service roles 
(both existing and evolving ones) and performance. 

The AAHSLD data must therefore reflect library inputs, processes, and outputs. In the future it would 
also be useful to review the current state of outcomes research and determine what additional res-arch 
would be needed to assess the impact of library resources and services on health science center outcomes. 
Empirical work in this area is likely to be time<onsuming and cosdy. 
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Figure 2 
Strategic Planning Context 



AAHSLD 
Database 



T 
I 
I 




I 
I 

I 

Library 
Management 
Information 
System 



85 



Van House/McClure/Hert 



AAHSLD Study 



Draft, October 23, 1991 



In place of a definitive assessment of library performance, health sciences libraries and their 
medical centers rely heavily on comparisons with other, "similar" organizations. Each organization 
determines its own reference group, which may change with the issue. Therefore, the database must be 
designed to allow libraries to identify their own comparison organizations and manipulate the data 
accordingly. For data to be compared across institutions, as the AAHSLD data are, the data must be 
comparable across institutions and over time. The measures and data collection methods must be 
uniform. 

The choice of measures to be produced from the AAHSLD database must be based on three major 
criteria. The first is the quality of the data. The second is the value of the data for decision-making 
and strategic planning. It is difficult to define a limited set of most-useful measures because, as 
circumstances and priorities change, the libraries' data needs change. The third major criterion is the 
level of effort required to collect data. When libraries report data to AAHSLD, they are collecting 
data for other libraries, not for themselves. Internal needs and priorities must predominate. Libraries 
cannot devote substantial time and effort to AAHSLD data collection. 

Several possibly contradictory needs for the measures and data elements to be included have been 
identified. The data elements reported in the AAHSLD database must be fairly stable, and varied 
enough to accommodate a wide variety of needs. The data collected must be those that the reporting 
libraries themselves wish to use, both internally and externally. And the measures included must keep 
up with changes in information technology, in the health sciences, and in the health sciences center 
environment, and with health science library functions. However, the level of effort required to collect 
the data must be minimal. Data collection methods must be easy and explicit; the more direction and 
help that libraries get in collecting data, the more likely they are to do so uniformly. 

The data to be included in the AAHSLD database and in the Annual Statistics should be 
determined by the audience's information needs. These needs need to be better articulated by the 
AAHSLD library community. The needs should be based not only on current library activities but future 
visions of what the AAHSLD library should be doing three years from now. The database and any 
resulting MIS system must be extremely flexible, to allow users to retrieve and analyze data to meet 
their particular needs. 

We can identify many possible measures of use to AAHSLD libraries. The study team believes it is 
more important to consider the current questions and problems for which data are needed as well the 
long-term need for data imposed by new strategic directions, roles, and functions of the AAHSLD 
libraries. The question of roles and strategic directions is being addressed by the part of this study that 
is concerned with the development of vision statements. 

The next phases of this study, in its treatment of measurement issues and the AAHSLD statistical 
system, should focus on: 

• The identification of current and enduring problems, and important health sciences library 
functions, roles, and strategic directions, and THEN on the data needed to make decisions 
related to these strategic directions 

• The development and validation of measures that focus on inputs, processes, outcomes, and 
linkages with service activities 
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)» The assessment of methods used to collect the current data and the development of easy 
methods for collecting high-quality data 

• The issue of which specific types of data should be collected for the AAHSLD data base and 
which should be collected for internal use by the library 

• The design of the database and library MIS to provide those data in a flexible system that 
would allow libraries to construct their own data sets and analyses to meet local needs. 

Within this framework, future phases of the project will need to integrate measurement, data 
management and MIS, and visions of library services into a comprehensive strategic planning process. 



NOTES 

1. Joanne G. Marshall, The Impact of Information Provided bv the Hospital libraries in the Rochester 
Area on Clinical Decision Making: Final Report . Faculty of Library and Information Science, 
University of Toronto, 1991; also reported in Robert J. Joynt, Joanne G. Marshall, and Lucretia W. 
McClure, "Financial Threats to Hospital Libraries" (Letter), lournal of the American Medical 
Association 266:9 (September 4, 1991) p. 1219-1220. 

2. John Bryson, Strategic Planning for Public and Non-profit Organizations. San Francisco: Jossey-Bass, 
1988, p. 5. 
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SUMMARY AND ANALYSIS OF MEDICAL CENTER ADMINISTRATORS 

FOCUS GROUP 

A Draft Report Prepared for the 
AAHSLD Editorial Board 

By 

Carol A. Hert, Charles R. McClure, and Nancy A. Van House 
For the Study 

A PLANNING, SERVICE ROLES, AND PERFORMANCE MEASURES 
MANUAL FOR ACADEMIC HEALTH SCIENCE LIBRARIES 

November 18, 1991 

PURPOSE OF THE REPORT 

After the analysis of the initial focus groups with academic health science librarians was 
concluded it became clear that input from academic health science center administrators would be 
necessary in order to understand the planning issues facing academic health science librarians. A focus 
group of medical center administrators was held November 8, 1991 in Washington D.C. at the 
Washington Hilton Hotel in conjunction with the Association of American Medical Colleges conference. 
Four administrators were present. An additional administrator was interviewed separately, and the 
comments of this administrator are integrated into this summary report Thus, the summary and 
recommendations are based on input from five individuals. 

The views and opinions of administrators on a number of topics were solicited. These topics 

were: 

• The current level of planning activities in the institution and the role of the library in 
those activities 

• The perception of library performance on campus 

• The ways in which the library justifies itself to the institution and the types of measures 
of quality and success which are appropriate 

• The criteria used to identify appropriate reference comparisons 

• The evolving visions and service roles for academic health science libraries 

• Ways in which the library could become a major institutional contributor 

This report presents a summary of the focus group and associated interview, specific issues which the 
study team identified through analysis of the data, and recommendations for continuing action. 

EXECUTIVE SUMMARY 

Participants in the focus group explored a number of topics. The key issues raised during the 
discussion were: 

• Administrators are as concerned with better measures of library performance as librarians. 
Measurement needs to focus more on user needs and impacts on the institution. 

• Comparative group comparisons are less important than outcome measures of library 
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performance for administrators. 

• There are a number of dimensions of the institutional setting which impact planning 
activities. 

• Administrators believe the library needs to become a more visible player in institutional 
planning. 

• The library needs to present a strategic vision of the future but this vision needs to be 
developed in partnership with other stakeholders. 

These issues present a number of areas for future action on the part of the study team. These areas were: 

• The continuing exploration of institutional settings to determine those factors which are 
important from planning from both the adminstration and library views. 

• The development of user-based outcome measures. 

• The development of performance measures that assess the library's impact on a range of 
institutional activities. 

• Incorporation in the planning manual of strategies and techniques for gaining and keeping 
involvement in institutional planning activities. 

• Ongoing education of librarians in institutional planning activities. 
Action in these areas will be considered in tasking for future phases of the project. 



SUMMARY OF THE FOCUS GROUP 

Library Involvement in Planning Activities 

Participants first commented on the level of planning in their institution and library 
involvement in those activities. All participants cited the existence of ongoing planning activities at 
the institutional level. Several different types of organizational configurations were mentioned (e.g. 
stand-along medical school, medical center existing in near proximity to other similar institutions, 
medical school integrated into a general university setting) and these configurations were indicated as 
important features of the planning environment for the institution. For example, the medical school 
which existed in close proximity to other medical centers was engaged in inter-institutional planning 
activities, while another participant stated that her medical school had to "stand on its own bottom," 
and was free-standing strategically and financially. 

On the whole, whatever the organizational configuration, the library is generally not an 
active participant in institutional planning projects. One administrator commented that while there 
are now 9 committees involved primarily with strategic planning (in the ares of money, space, 
community service, etc.), the library was not included on any of them. The library did have 
representation on the information technology committee and there was a separate library committee as 
well. Another administrator said that was the situation at his institution as well, saying that the 
library committee itself was well attended and faculty are interested in the library but that the 
library is poorly represented in other planning activities. One of the reasons the library is not 
represented on planning committees in his institution is that there are no faculty members in the library 
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to serve on such committees. One administrator said that within her institution there were several 
different planning efforts and the library was considered in these efforts to the same extent as other 
departments. 

The Perception of Library Performance at the Institution 

The administrators held varying perceptions of the role of the library in the institution. One 
participant commented that the library thinks of itself as always being there so that it thinks it 
doesn't have to demonstrate its value. This comment was echoed by another participant who said that 
the library seems a dull place. It hasn't managed to articulate its need for institutional support and 
doesn't justify why money should be spent on it. A slightly less extreme perception came from another 
participant who said that the library is universally accepted as a good resource on campus and is seen 
as being in transition. However, the librarians and others in the institution seem unclear on what 
services, etc. the library should be providing. Another said the library was widely admired on campus. 

How Does the Librcjy Justify Itself to its Institution? What Types of Measures of Quality or Success 
Would be Appropriate? 

Just as the librarians in the focus groups mentioned, administrators, too, struggle with 
appropriate means of determining the library's value to the institution. One administrator cited a case 
where the institution was trying to decide whether librarians should be on duty until 11:00 pm and 
couldn't determine what kind of arguments would be persuasive. At that institution the arguments that 
are frequently acceptable to the institutional administration are that the library cannot fall behind its 
competition and that acquisitions cannot fall because then the library could not guarantee that 
resources would be available when necessary. Another participant said the library used Annual 
Statistics data as arguments but that he didn't see those data as having much credibility. 

When asked whether comparative data was important, the administrators had differing 
opinions. One administrator said that she bought the argument that the library had to keep up with 
the competition (particularly in the area of acquisitions) but that she wasn't sure whether the data 
were valid. Two administrators argued that comparative data on acquisitions was not important, that 
what was needed was a more user-based measure such as availability since given increasing networking 
ability it doesn't matter if the item is in the collection or not as long as it is accessible through the 
network. An important point brought out by the administrators was that they are aware that the 
library changes peer groups and comparisons presented based on the argument that is being made. This 
action tends to make them skeptical of the value of comparative data. 

The discussion focussed on acquisitions for a moment and participant comments on the topic 
reinforced the perception that different medical centers have different "institutional cultures" which 
determine acquisitions patterns (and probably other library features) and therefore arguments which 
can be made to administrators. One participant commented that there were many duplicate 
subscriptions in the center (partially due to an abundance of funds earmarked for specific purposes or 
projects) and that no one would cancel a personal subscription but everyone still felt the library should 
have the copy of record. One participant described his institution as a confederation of fiefdoms with 
each department trying to act independently. Another participant said that at her smaller, less well 
funded institution, there is not that level of decentralization. 

When asked how could you tell when you had a good library, participants had no good 
measure. One could tell when the library wasn't functioning properly when complaints were registered. 
At one institution, there was a 5% cut across the board and the library cut hours and subscriptions which 
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raised an incredible uproar. All participants agreed that complaints were what got their attention and 
affected their decisions regarding the library. One administrator commented that if they received 
complaints about the library they knew they had cut its budget too much. If the administrators don't 
hear anything, the library probably had the appropriate level of funding, and if they are saying good 
things, mat means the library has too many resources. Another participant reminded the group that 
one has to be careful of complaints; change always generates complaints (e.g.; the closing of the card 
catalog) so complaints may not be a good measure of library performance. 

There was agreement that better measures of library effectiveness needed to be developed. One 
participant cited the increasing demands on all departments to be responsive to institutional 
management needs and to be able to provide relevant and easily integrated information on performance. 
He felt that management units outside the library would develop such measures and statistical 
benchmarks for the library if it does not develop its own. 

Administrators enumerated a number of measures and types of data which they would find 
persuasive. These were: 

• Accessibility of information (not collection size) 

• Ease of obtaining information from the library 

• Quality of information available from the library judged in terms of what information is 
available in the larger universe 

• Library responsiveness to clientele needs 

• Library ability to anticipate clientele needs 

• Number of people connected electronically to the library 

• Correct answer rate (if a standard could be developed) 

• Comparative anecdotal and testimonial data 

• Time saved on the part of the physician 

• Contribution to research 

• Return on investment (what did the library provide against its costs?) 

One administrator commented on the relationship between measures and library management 
information systems. Information derived from an MIS and from statistics should be able to 
demonstrate a return on the institution's investment of resources. The MIS needs to provide some kind of 
evaluation of available institutional information resources (including human resources). The 
administrator also though the MIS should be designed to be flexible enough to evaluate various aspects 
of issues and problems including new issues and problems. Finally, he asked whether the library could 
build academic credibility by developing a model MIS and then assisting other departments to develop 
their own. 

What Criteria are Used to Identify Appropriate Reference Comparisons? 

Administrators pointed to a number of factors used to determine like schools" for comparison 
purposes. One administrator commented that you generally knew what schools to compare to. Factors of 
importance in determining a comparison school that mentioned by participants were: 

• Whether the institution was research intensive or teaching oriented 

• Whether the comparison school competed with the institution for faculty or post-docs 

• Whether the school was community based (relying on faculty in the community hospitals) 

• Whether the school was part of a consortium 

• Whether the school was public or private 

• What was the school's mission (e.g., training primary care physicians or academic 
physicians) 
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• What other health delivery schools are affiliated 

• Whether the institution was on a main university campus or not 

• Urban or rural 

• Reporting line for library 

These factors add to those mentioned by academic health science librarians during other focus group. 

What Are the Evolving Visions and Service Roles for Libraries? 

The administrators were concerned that the library develop a more visionary stance within the 
institution. The administrators wanted to be exdted by the library. Many of the visionary ideas 
suggested by the administrators included the move towards more electronic access to information. Some 
of the participants felt there library was trying to take a leadership role but others thought that the 
library was behind the rest of the institution in this area. One administrator thought the library 
should stop ordering materials and should just provide access. One person suggested that the library of 
the future should just be a switching computer. Another new service role mentioned by the 
administrators included increased library involvement in computeriyed courseware development along 
with ongoing work in computer literacy. 

One administrator reacted negatively to his library's creation of a service marketing position 
within the library. He perceived this as trying to sell unwanted services rather than providing new 
services that are necessary and desired by users. Another administrator felt that is was a useful 
strategy to educate administrators, faculty, and students as to the positive benefits that can accrue to 
them if the institution places more resources in the library. The library should also get the attention of 
the people who control institutional resources. 

One administrator cited a number of trends which are influencing the ways in which the 
library relates to the larger institution)! setting. These trends were: 

• Information is increasingly available in electronic formats. 

• Faculty and staff need to re-tool in order to use electronic iivformition effectively. 

• The cost and amount of information continues to increase. 

• Libraries will be forced to plase increasing emphasis on access rather than ownership. 

• Libraries will be challenged to find new and better ways to share resources and establish 
cooperative ventures. 

• Libraries will be called upon to do more quality filtering to reduce the amount of 
information to that which is needed. 

• increasing specialization of health care researchers and providers and the information 
ithey generate may exert pressure for librarians to become more specialized, concentrating 
on the needs of specialized clientele. 

The administrator felt these trends would result in a number of new ways in which the library could 
provide service Jo the institution. Among these ways were: 

• Involvement in the selection of information sources appropriate to the institutional 
network 

• Provision of assistance to faculty and students in re-tooling for new information technology 

• Enhancement of the teaching and consultive roles of the library 

One of the most important ways the library could servo the institution, according to this administrator, 
was through its involvement in institutional management processes. He felt that informed management 
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decisions will require increasing assistance in obtaining, analyzing, reducing, summarizing, and 
interpreting information. With the increasing need for academic health science centers to become more 
competitive, they may need to spend more money on libraries to obtain quality information in order to 
contain institutional costs. 

What can the Library do to Become a Major Contributor to the Institution? 

The concluding discussion focussed on the question of what the library could do to become a major 
contributor to the institution. Participant comments centere d on the need for increasing involvement in 
computer initiatives on campus. One administrator said that the library had lost an opportunity by not 
being involved earlier in this realm. One administrator suggested that mere needs to be a changed 
relationship of individuals and organizational units charged with bringing about the implementation 
and integration of technology with the library as an integral component of institutional information 
policy setting and infrastructure. 

Library involvement in networking and computing initiatives needs to be guided by a strategic 
vision. All the administrators agreed that this was a vision that the entire institution (faculty, 
students, library, community) could work on together as a team. One participant pointed out that 
libraries needed to focus on student and faculty needs since they are the consumers. 

Along with establishing a more visionary stance, the administrators also agree that the 
library needed to insert itself more aggressively into planning activities within the institution. The 
library needs to be an active participant in planning, choosing options, and decision making. One 
administrator pointed out that intra-institutional competition for resources will be increasing and 
resources will be allocated to those organizational units which have good plans and planning 
strategies. Therefore it behooves the library to focus attention on planning activities. As another 
administrator put it "Join the revolution, start planning and adapt. Effect a transition from the 
traditional library and let's create a vision together." 

ISSUES RAISED 

The focus group raised a number of important issues for the planning activities of academic 
health science libraries. These were: 

• Administrators are as concerned with better measures of library performance as librarians. 
Measurement needs to focus more on user needs. 

• Comparative group comparisons are less important than outcome measures of library 
performance for administrators. 

• There are a number of dimensions of the institutional setting which impact planning 



• Administrators believe the library needs to become a more visible player in institutional 



• The library needs to present a strategic vision of the future but this vision needs to be 
developed in partnership with other institutional stakeholders. 

Administrators are as concerned with better measures of library performance as librarians. 
They feel stymied by the lack of measures which will provide useful information for decision making. 



activities. 



planning. 
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The administrators felt that measures which were more user-based needed to be developed, that is, 
measures which determine whether the outcomes experienced by users are acceptable. They mentioned 
several measures which would be useful to mem as administrators. These were: 

• Accessibility of information (not collection size) 

• Ease of obtaining information from the library 

• Quality of information available from the library judged in terms of what information is 
available in the larger universe 

• Library responsiveness to clientele needs 

• Library ability to anticipate clientele needs 

• Number of people connected electronically to the library 

• Correct answer rate (if a standard could be developed) 

• Comparative anecdotal and testimonial data 

• Time saved on the part of the physician 

• Contribution to research 

• Return on investment (what did the library provide against its costs?) 

These measures complement those mentioned by librarians in both the focus groups and the workshop. 

Generally, the administrators felt that outcome measures are more valuable for decision 
making than comparative measures between libraries. They were aware that liN-aries frequently 
change the peer groups and the measures used in order to bolster particular arguments. This practice 
has led administrators to rely less heavily on such measures. Such measures also do not provide enough 
information for administrators to evaluate the library's value to the institution. Outcome measures 
need to be developed to provide better information for institutional decision makers. 

As the project has developed, it has become increasingly clear that institutional setting plays a 
key role in determining the extent and success of library planning activities. The administrators 
mentioned the following dimensions of the setting.which they felt affected the choice of peer groups 
and influenced decision making. 

• Whether the institution was research intensive or teaching oriented 

• Whether the comparison school competed with the institution for faculty or post-docs 

• Whether the school was community based (relying on faculty in the community hospitals) 

• Whether the school was part of * consortium 

• Whether the school was public or private 

• What was the school's mission (e.g., training primary care physicians or academic 
physicians) 

• What other health delivery schools are affiliated 

• Whether the institution was on a main university campus or not 

• Urban or rural 

• Reporting line for library 

These dimensions need to be considered in future phases of the project along with dimensions which 
librarians indicated as being important. 

Administrators seconded the librarian's perception that the library needs to be more actively 
involved in institutional planning activities. While they believed that the library needs to be more 
involved, they did not suggest 'hat the administration should invite the library in, rather the library 
should be aggressive about getti ^ involved. This may be difficult to accomplish given institutional 
barriers such as committees consisting only of faculty members when most library staff are not given 
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faculty status. 

One possible foothold for the library in institutional planning may be in the joint development 
of visions for the library. The administrators felt the library needed a vision to shoot for and had a 
number of suggestions What administrators wanted was to be involved in the vision development 
process. Joint development would help insure that the vision made library and institutional needs and 
could provide an entry into institutional planning activities on a broader scale. 

FUTURE ACTION 

The issues raised in mis focus group suggest several areas for action on the part of the study 
team. These areas are: 

• The continuing exploration of institutional settings to determine those factors which are 
important from planning from both the adminstration and library views. 

• The development of user-based outcome measures. 

• Incorporation in the planning manual of strategies and techniques for gaining and keeping 
involvement in institutional planning activities. 

• The development of methods to establish peer group comparisons that are seen as credible 
by institutional administrators. 

• Ongoing education of librarians in planning activities. 

These areas will be considered in tasking for additional phases of the project. 

As mentioned above, administrators added to the list of institutional setting dimensions which 
may be important in planning activities. The study team needs to explore whether other dimensions 
exist, and which are most important. These dimensions would need to validated. 

Findings from this focus group add to the already overwhelming evidence that performance 
measures, particularly outcome measures need to be developed. Administrators were particularly 
concerned that the measures focus on the needs of the user. User-based measures which are useful to 
decision making on the part of the library and the administration need to be developed and validated. 

One of the clear points made by the administrators is that the library needs to be more 
involved institutionally but that they were not offering to assist the library in that area. If the 
library wants to be more involved it will have to actively seek that involvement. Findings from other 
focus groups suggested that library staff need help in identifying techniques to gain and maintain such 
participation. These techniques and strategies should be incorporated into the planning manual. 

Another important area for future action is the development of methods for determining 
appropriate peer groups. Administrators are aware that peer groups are frequently chosen to cast the 
best light on library activities. Work needs to occur on what would allow peer group comparisons to 
become more credible evidence for institutional administrators. 

Finally, ongoing education of AAHSLD librarians needs to occur. For the library to be 
successful, it will need better measures of its value to the institution, a clear vision, and increased 
involvement in institutional planning. The skills necessary to accomplish these tasks need to be 
developed through educational activities. 
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PLANNING, CSRVICE ROLES, PERFORMANCE MEASURES, AND 

MIS 

FOR ACADEMIC HEALTH SCIENCE LIBRARIES 

November 9, 1991, 9:00 AM - 4:00 PM 
Omni Georgetown Hotel, Washington D.C. 

Conducted By 

Charles R. McClure, Nancy A. Van House, and Carol Hert 
Information Management Consultant Services, Inc. 
Manlius,NY 13104 

Sponsored By 

Association of Academic Health Sciences Library Directors 



In preparation for this workshop, we would like you to review the following 
material. The discussion questions will be done at the workshop; you do not need 
to complete these now . We will explain the nature of the exercise in greater detail at 
the workshop. Abo enclosed is an abstract describing this project. 

Please remember to bring this handout with you to the workshop. THANKS! 



VISION STATEMENT CRITIQUE EXERCISE 

During the workshop, we will be exploring the utility of the following vision 
statements for academic health science libraries as well as asking you to develop 
your own. Before the workshop, we would like you to read over the enclosed 
vision statements, and consider the following questions for each statement: 

1. Does the vision statement adequately represent a vision which is achievable for 
an academic health science library? Why or why not? 

2. Is the vision one which you could imagine adopting in your library? Why or 
why not? 

3. Are the library assumptions listed adequate for accomplishing the vision? Why 
or why not? 
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ELECTRONIC NETWORKED LIBRARY 
Environmental Assumptions 

There is continued rapid growth and technical developments in high-speed 
telecommunications. Costs for various types of computer and telecommunications 
services decreases over the next three years. Library users become increasingly 
sophisticated about the uses and applications of computers and remote access to 
information resources. The National Research and Education Network (NREN) is 
implemented, giving academic faculty and staff access to a wide range of 
information services and resources. 

Example Library Goals 

The Library serves as a link between the community and the electronic networked 
environment; it serves as an "electronic navigator" in directing users (both from 
their home and at the Library) to the expanse of remote information resources and 
services. The Library's own information resources are largely available in machine- 
readable form to networked users from throughout the network. The Library 
educates its community of users and administration about the benefits and 
importance of being "wired" to the networked environment. 

Example Library Activities 

There are electronic bulletin boards for the community to access as well as bulletin 
boards to inform Library staff of current activities. These bulletin boards and other 
Library information resources and services are available via remote access by anyone 
in the community with a computer and modem. The Library's online catalog is 
searchable remotely by users, reference services are provided electronically and 
remotely, and information and documents are transmitted to users electronically. 
Program /discussion groups are developed and moderated over the network by 
Library staff. Library staff access the network and communicate regularly with each 
other about a range of Library matters, services, and administrative concerns. 

Library Assumptions 

The Library and its branches are able to obtain appropriate computer equipment, 
establish and participate in local area networks (LANS) to communicate with each 
other, and to access regional and national electronic networks. The Library has staff 
able to plan and implement the use of the networks. Staff are trained in the uses 
and applications of the new technologies. Library buildings are re-organized to 
provide for both public and staff accc s to the computer/telecom workstations. 
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INFORMATION SUPPORT CENTER 
Environmental Assumptions 

The institution continues to look to the Library as the source of traditional Library 
resources and services. There is also an increasing demand from clientele for value- 
added services and for information management services. 

Example Library Goals 

Comprehensive information support services are given to a broad range of clientele 
(e.g researchers, clinicians, faculty, allied health personnel and students). The 
Library strives to improve these services through the innovative use of technology. 
The Library facilitates the exchange of ideas between clientele. Library services and 
resources are promoted to the clientele and suggestions for new services solicited. 

Example Library Activities 

The research and educational information needs are supported »s fully as possible 
through ready-reference service, in-depth consultation, and departmental and 
specialty programs. The Library develops information products, including 
knowledge and information databases and other packaged information. Library staff 
provide instruction and assistance in the identification of appropriate resources 
(through online catalogs, CD-ROM's, remote databases, etc.), in the use of resources, 
and in information management techniques (e.g. personal file management). If 
appropriate, this instruction is integrated into the formal course offerings of the 
institution. Self-pared instructional materials are available. Questions and other 
requests are received in person, over the telephone, and through electronic 
communications. Information and consultation services are delivered directly to 
Library clients using the latest computer and networking technologies. 
Consultation services include selective dissemination of information. Library staff 
work closely with individuals to assess their information needs and to locate, access, 
and synthesize potentially relevant materials. 
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Library Assumptions 

The Library develops and maintains strong collections in areas of interest to the 
clientele. Adequate facilities to house the collection are available. Facilities for 
study, research, and teaching are available. All CD-ROM's, remote databases, and 
networks needed for instruction are available. Access to external resources is made 
possible through effective inter-library loan service and electronic networking. 
Library staff is skilled in the provision of services of all types and thinks creatively 
about the extension of these services. Staff is trained in the latest technologies and 
equipped to provide instruction in their use. 
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INFORMATION RESOURCES MANAGER 
Environmental Assumptions 

There is continuing and increasing need for effective planning at the institutional 
level. Units of the institution are increasingly interconnected, necessitating more 
communication and involvement of all parties in institutional planning. Access to 
information resources, information management tools, and decision-making tools 
is increasing important to effective management. 

Institutional resources of all types remain limited. This necessitates the reduction 
in duplicate acquisition of information resources and associated tools and well as 
the development of efficient methods of coordinating the process of acquiring and 
utilizing those resources. 

Example Library Goals 

The Library is an active participant in the management, planning, and evaluation 
activities of the institution. The Library coordinates the selection, purchase, 
installation, and maintenance of information resources in all formats and associated 
management tools for the institution as a whole. The Library provides training in 
the use of resources and tools. 

Example Library Activities 

Library staff is actively involved in the management, planning, and evaluation 
activities of the institution through their service on committees, task forces, and 
other planning bodies. The Library and Library staff provide information resources 
and expertise to facilitate these activities. Fast, accurate answers are provided in 
response to the information needs of administrators. Administrative information 
management tools (e.g. spreadsheets and MIS) are available for use. Instruction in 
the use of the tools is provided. 

The Library coordinates the purchase of bibliographic materials in all formats and 
the selection, purchase, installation, and maintenance of management tools 
including word-processing, database management, and other information 
management software packages. The Library uses acquisitions and inventorying 
software to manage these functions. The Library subscribes to necessary and desired 
periodicals and utilizes sophisticated serials controls systems to provide routing and 
inventory functions. Information about bibliographic materials located throughout 
the institution is accessible though the Library's online catalog. Training is 
provided in the use of the resources. 
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The library has sufficient space to store necessary information resources and well as 
work space for staff involved in acquisition, inventorying, and warehousing 
functions. Library staff is trained in the installation, maintenance, and use of 
information management hardware and software products. Library staff have 
expertise in the areas of resource selection, use and are able to train users. Facilities 
are equipped with the technology necessary to demonstrate and use information 
management tools. 

The Library is able to obtain, maintain, and upgrade appropriate software and 
sufficient computer equipment to support the necessary tools. The Library has 
access to necessary information resources either through in-house availability, inter- 
library loan services, and networking to external resources. The Library staff is 
trained in the use of the technology, management techniques, and the use of 
information resources. The Library and its staff is recognized as integral 
components of institutional decision-making processes. 
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EDUCATIONAL AND CURRICULUM SUPPORT CENTER 
Environmental Assumptions 

The institution is increasingly involved in initiatives both in the area of medical 
informatics, research, and in the use of computers in the curriculum. The 
evolution of integrated information resources within the academic health science 
campus allows the Library to utilize its resources more directly in the teaching 
function. Moreover, academic departments increasingly realize that they cannot 
develop curriculum effectively without the integration of both traditional and 
electronic information resources-many of which are owned or could be accessed by 
the Library. 

Example Library Goals 

The Library provides access to instructional technology and computer-based 
curriculum materials. The Library is proactive in the design, selection, and 
application of instructional technology and computer-based curriculum materials. 
Library staff assist instructors in the use of these materials. The Library facilitates 
change in the curriculum (e.g. the move to problem-based instruction) through the 
provision of appropriate information resources and consultation services. The 
Library plays an active role in institutional medical informatics initiatives. 

Example Library Activities 

The Library provides audio-visuals, computer-based educational packages, and other 
educational technologies for use of site and for loan. It may be responsible for an 
instructional technologies lab and work with other instructors in the integration of 
such technology into the curriculum. The Library uses technology to make it 
possible for members to access educational technology products from remote 
locations. Assistance is provided in the selection, use, and development of tools. 

The Library is an active partner in medical informatics initiatives within the 
institution, remaining abreast of the latest developments in the field and by 
providing access to and assistance with using related information resources. Library 
staff collaborates with faculty in the development of innovative educational 
technology and medical informatics products to support the missions of the 
institution. 
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Library Assumptions 

Facilities and tools for development of educational technology are available. 
Facilities and equipment appropriate for use in developing medical informatics 
products are available. Technology is in place to support remote access and use of 
the collections. Library staff is able to maintain and upgrade necessary equipment. 
Library staff is skilled in the development of educational technology and medical 
informatics products. The Library remains abreast of advances in theory, practice, 
and technology in support of curriculum development. There is appropriate and 
sufficient space for consultation. Library staff is skilled in educational techniques 
and information services. 
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INFORMATION TECHNOLOGY INNOVATOR 
Environmental Assumptions 

The development of various information-handling technologies continues at an 
unprecedented pace. The cost for these technologies decreases. The Library is a 
central force on campus to maintain and exploit a range of information 
technologies. These technologies include computer, telecommunications, satellite, 
and other evolving communication technologies. The institution is interested in 
determining how these information technologies can be linked and applied to 
accomplishing various institutional goals. 

Example Library Goals 

The Library serves as a model and demonstration of the use of innovative 
information technology in the areas of information retrieval and access, 
information resource management, and adminstration. The Library remains 
abreast of the latest developments in information technology in these areas and 
implements those that are appropriate. The Library maintains a presence in 
regional and national organizations in order to facilitate the use and sharing of its 
expertise in information technology. Library staff performs related research. 

Example Library Services 

Currently operating systems in the areas of information retrieval and access, 
information resource management, and administration are in use in the Library 
and are able to be demonstrated to interested parties. These may include expert 
systems, CD-ROM databases, advanced user interfaces, management information 
systems, and acquisitions and inventorying systems. Library staff provides training 
in the use of available tools. Library staff evaluates the effectiveness of these 
systems and presents results of such studies at regional and national meetings. 
Research into ongoing improvement in design of such systems is undertaken. 

Library Assumptions 

Facilities are available to house necessary equipment and software. Classroom and 
seminar space is available to demonstrate and educate in the use of various 
technologies. There is sufficient space for staff to consult with interested parties 
about information technology needs. Library collections include the necessary 
software and reference manuals and documentation. Library staff is trained in 
information technology use. Library staff is capable of installing, maintaining, and 
upgrading equipment and software. 
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CLINICAL INFORMATION RESOURCE CENTER 



Environmental Assumptions 

The health science community increasingly turns to a clinical perspective on health 
care research and delivery. This perspective is one in which on-site learning and on- 
site information services delivery is essential. Out-patient services, emphasis on 
"wellness," and direct and regular interaction among (1) patients, (2) the medical 
care delivery units, and (3) the Library staff have been found to provide the best 
organization of resources for health care services. 

Library Goals 

The Library and its staff play a critical role in the clinical activities of the institution. 
Library Activities 

Library staff spend considerable time outside the physical setting of the Library and 
are involved directly in the support of various health care services. This support 
may take the form of on-site information delivery, providing immediate and direct 
access to global information resources, designing information services for unique 
clientele, or evaluating the effectiveness of current information services. 

The health science clinicians on campus are aware of the range of information 
services support that the Library can provide. They have Library staff participate in 
program development, in work/strategy sessions, and in on-site delivery of health 
care services. In information resources/services training activities, Library staff 
work directly with the clinical staff and students. 



Library Assumptions 

Library staff is knowledgeable about program design, implementation, and 
evaluation. The Library has excellent relationships with the clinical programs on 
campus. Librarians are able to conduct needs assessments for the information 
services and resources that might be required by different clientele groups. The 
Library is able to support a team of mobile librarians who can work at various 
locations around campus as well as in the local community. The Library has 
networking capabilities to facilitate communication between the Library, clinical 
librarians, and others in the clinical programs. 
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PLANNING, SERVICE ROLES, PERFORMANCE MEASURES, AND MIS 
FOR ACADEMIC HEALTH SCIENCE LIBRARIES 

November 9, 1991, 9:00 AM - 4:00 PM 
Omni Georgetown Hotel, Washington D.C. 

Conducted By 
Charles R McQure and Nancy A Van House 

Sponsored By 

Association of Academic Health Sciences Library Directors 



Description and Objectives. The workshop will provide an overview of strategic 
planning, performance measures, service roles, and management information 
systems (MIS) for increasing organizational effectiveness in academic health science 
libraries. Specific objectives of the workshop include: 

• Describe preliminary results from the AAHSLD funded study "A New 
Strategic Direction for the Annual Statistics: Development of A Planning, 
Service Roles, and Performance Measures Manual for Health Science 
Libraries." 

• Introduce participants to the importance and use of planning, vision 
statements, performance measures, and MIS as a comprehensive 
management strategy. 

• Consider possible planning strategies, academic health science performance 
measures, and explore new service roles for academic health science libraries. 

• Conduct a number of group discussion sessions for participants to provide 
input to the study and to assess preliminary findings. 

By the completion of the workshop, attendees will have a better understanding of 
the role and importance of strategic planning, performance measures, and 
management information systems for academic health science library management. 
Moreover, they will have offered perspectives and information that will direct 
future phases of the project. 
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Tentative Schedule: 

830 - 9:00 Registration and Coffee 

Welcome and Introduction to the Workshop 



9tf0-9:10 
9:10-9:45 

9:45-10:40 

10:40-11:00 
11:00- 11:15 

11:15 -Noon 

Noon - 1:00 
1:00-1:30 



130-2:30 

230-2:45 
2:45-3:00 
3:00-3:30 
330-3:45 
3:45-4:00 



Overview of the AAHSLD Project and Key Issues in Academic 
Health Science library Planning, Service Roles, Performance 
Measures, and Managing Data for Decision Making 

Group Exercise and Discussion of Evolving Vision Statements 
and Service Roles in Academic Health Science Libraries 

Break 

Institutional Settings for Strategic Planning, Evaluation, and 
Data Management 

Group Exercise and Discussion of Institutional Settings and their 
Relationship to Library Strategic Planning 

Lunch 

Developing Performance Measures 

• traditional library performance measures 

- possible performance measures from the AAHSLD database 

- new performance measures for Academic Health Science 
Libraries 

- linking performance measures to service roles 

Group Exercise and Discussion of Developing and Assessing 
Performance Measures for Vision Statements and Service Roles 

Break 

Managing Performance Measure and Strategic Planning Daia 
Demonstration of TMC Management Information System 
Discussion of MIS Demonstration 

Next Steps for Developing Academic Health Science Library 
Strategic Planning, Performance Measures, Services Roles, and 
MIS 
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GROUP EXERCISE: VISION STATEMENT CRITIQUE 



Before the workshop, we asked you to review several possible vision statements for 
academic health science libraries and critique them by considering a number of 
questions: Did the vision statement adequately represent a vision 'hich is 
achievable for an academic health science library? Was the vision one which you 
could imagine adopting in your library? Were »hc library assumptions listed 
adequate for accomplishing the vision? Were the environmental assumptions the 
relevant ones? 

During mis exercise, we would like you to consider as a group whether the vision 
statements are appropriate. Rich group should hand in one summary page 
indicating group member thoughts about each statement. For each vision statement 
answer the following questions: 

Does the vision statement adequately represent a vision which is achievable 
for an academic health science library? Why or why not? 

Electronic Networked Library 



Information Support Center 



Information Resources Manager 



Educational and Curriculum Support Center 



Information Technology Innovator 



Clinical Information Resource Center 
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GROUP EXERCISE: VISION STATEMENT DEVELOPMENT 



A vision statement is a description of a possible future state or set of functions for an 
academic health science library. To develop a vision statement, you need to make 
explicit your assumptions about the future, envision a future state of the library in 
light of those assumptions specifying relevant goals and possible activities, and 
consider organizational goals and resources. Use the form below to revise one of 
the vision statements or to develop a new one for your library. Each group should 
hand in one vision statement to the instructors. 

VISION TITLE: 

Environmental Assumptions 



Example Library Goals 



Example Library Activities 



Library Assumptions 



Select one vision statement for which the group will develop performance 
measures in the afternoon session. 
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GROUP EXERCISE: ASSESSING INSTITUTIONAL SETTINGS: PART 1 



Planning occurs in an institutional setting. For planning to be effective, planners need 
to assess the importance of institutional features. In the first part of this exercise, we 
would like you to develop a description of your institutional setting. Consider the 
following questions as you develop the description. 

Institution Name 

1. Who makes the top decisions in your organization? 

2. What is their professional background? 

3. What kinds of information about the library do they request? 

4. What are the important institutional missions? 

5. What is the importance of teaching, research, and patient care in the institution? 

6. What are major strategic issues facing your organization? Facing the library? 

7. What is the commitment to planning in the library? In the institution? Are 
you doing planning in specific areas? If so, which? 

8. How knowledgeable about planning and management are people in the library? 
in the institution? 

Your Library's Institutional Setting 

Use the back of the page to describe the setting for your library in 3-5 sentences. 
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GROUP EXERCISE: ASSESSING INSTITUTIONAL SETTINGS: PART 2 



As i group, we would like you to consider the commonalities you observe in the group 
members' institutional settings. Answer the following questions and hand in one copy 
to the instructors. 



1. What are thekey similarities and differences you observed in group members' 
institutional settings? 



2. Within the group, can you describe 2-3 typical institutional settings? 



3. Rank the factors listed in question one in order of importance, with the most 
important first. 



4. In each of these institutional settings, how might the library create a more 
receptive environment for successful planning? 
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GROUP EXERCISE* DEVELOPING PERFORMANCE MEASURES 

This exercise will focus on developing measures which can be used to determine the 
degree to which a vision has been attained, whether progress towards a vision is 
occurring, as well as be used to justify library performance to medical center 
administrators. After reviewing your vision statement, answer the following 
questions. The group should hand in one copy at the end of the exercise. """ 

Vision Statement Title: 

1. What type of outputs or outcomes are necessary to determine whether you have 
attained the vision? 



2. What types of outcomes are necessary to evaluate progress towards a vision? 



3. What types of measures could be developed to demonstrate those outcomes? 



4. What types of internal data would be needed to produce the measures? What 
types of comparative data? 



5. What sources of internal data are available within your institution to produce 
these measures? What sources of comparative data? 
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PLANNING, SERVICE ROLES, PERFORMANCE MEASURES, AND MIS 
FOR ACADEMIC HEALTH SCIENCE LIBRARIES 
WORKSHOP EVALUATION FORM 

The organizers are interested in your evaluation of today's workshop. Please take the time to complete the 
following questionnaire. Leave it in the box at the exit Thank you. 

1. How would you rate the workshop overall? 

ExceOant Good Fair Poor 

2. Whit were the two best features of the workshop? Please explain. 



3. What two features could have been improved? Please explain. 



4. Which sections were most useful to you? (Please circle the best response.) 



Very Useful Not Useful 

Overview of the AAHSLD project 5 4 3 2 1 

Issues in planning, service roles, use of performance 5 4 3 2 1 

measures and managing data 

Academic sealth science library service roles 5 4 3 2 1 

Institutional settings for planning, evaluation, 5 4 3 2 1 

and data management 

Developing performance measures 5 4 3 2 1 

Managing per f o r mance measure and strategic planning 5 4 3 2 1 

data 

Demonstration of TMC Management Information System 5 4 3 2 1 



5. How would you rate the physical facilities? 

ExoeCant Good Fair Poor 

6. What additional educational follow-ups on these topics would you recommend? 



7. Please use the back of the form for additional comments. 
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Summary of PLANNING, SERVICE ROLES, PERFORMANCE MEASURES, 
And MIS FOR ACADEMIC HEALTH SCIENCE LIBRARIES Workshop 

A Draft Report Prepared for the 
AAHSLD Editorial Board 

By 

Carol A. Hert, Charles R. McClure, and Nancy A. Van House 
For the Study 

A PLANNING, SERVICE ROLES, AND PERFORMANCE 
MEASURES MANUAL FOR ACADEMIC HEALTH SCIENCE LIBRARIES 

November 18, 1991 



PURPOSE OF THE REPORT 

As part of this study, a workshop was held on November 9, 1991 at the Georgetown Omni Hotel 
in Washington, D.C. This workshop had several objectives. The primary objectives were: 

• To increase awareness of the project on the part of the membership. 

• To inform member s hip of the components of planning and focus attention on the planning 
process for academic health science libraries. 

A secondary objective was to provide an opportunity for the study team to assess the extent to which 
previous data collection activities had contributed to the study team's understanding of the planning 
process for AAHSLD member libraries and to receive input from workshop participants which could be 
used to guide further efforts. Meeting this objective is critical to the development of a planning manual 
which will be useful to academic health science libraries. 

This report presents a summary of the workshop and includes a preliminary analysis of the 
workshop evaluations, summary findings from each of the group exercises, as well as a discussion of 
issues which arose in the workshop and their implications for future data collection activities. 

EXECUTIVE SUMMARY 

Workshop evaluations indicated that the workshop met the set objectives. The AAHSLD 
membership was made aware of the project status and were informed about the components of the 
planning process and their interconnection. Additionally, die workshop allowed the study team an 
opportunity to present some preliminary findings and assess the extent to which the team understood 
the factors which impacted on planning in the member libraries. 

The three group exercises provided attendees with an opportunity to interact and explore vision 
statement creation, institutional setting definition, and performance measure development. Key 
conclusions drawn by the study team after analysis of the group exercises are: 

• The draft vision statements presented to workshop participants should be more visionary. 
Participants provided useful information for their refinement and raised some key questions 
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about the nature of the process. 

• Participants were able to describe their unique institutional settings. A number of new 
dimensions of settings which need to be considered in the planning process were elicited. 
Several dimensions were considered very important to the planning process including 
reporting line of the library, whether the institution was public or private, the availability 
of resources, institutional priorities, and the clientele served. 

• Participants were able to develop outcomes and performance measures linked to particular 
vision statements. There was some confusion about the difference between an output and an 
outcome measure. 

These findings led the study team to some recommendations for further action. Areas for further action 
are: 

• Continuing exploration of the linkages between institutional settings, vision statements, 
performance measures, and management information systems (MIS) in order to facilitate 
planning in AAHLSD member libraries. 

• Refinement of the dimensions of institutional settings which affect planning activities. 

• The development of a model of the vision statement creation and utilization process. 

• The ongoing education of AAHSLD member s about planning, performance measures, and MIS. 
These recommendations for action will be included in the project tasking for future phases of the project. 



WORKSHOP EVALUATIONS 

83 of the 110 workshop participants completed an evaluation form at the conclusion of the 
workshop. The evaluation form (Attachment 1) asked participants to rate the workshop overall as 
well as comment on those activities which were most useful or which could be improved. 

78 participants responded to the question asking for an overall rating of the workshop. On a 
scale of 4 (excellent) to 1 (poor), the mean rating was 3.16 indicating general satisfaction with the 
workshop. 

When asked to comment on the best features of the workshop, respondents mentioned all 
components of the workshop including the small group discussions, the types of exercises, and the 
content of the sessions. Suggestions for impr o vement included changing the room and providing more 
breakout rooms, allowing longer time to report back from group discussions, changing group makeup for 
each exercise to allow increased interaction between participants, and providing more content on 
selected topics. 

Participants were also asked to evaluate how useful the topics were to them On a five point 
scale with 5 being most useful and 1 being least useful, the means for each topic were: 

Overview of the AAHSLD project 3.8 (N-78) 

Issues in planning, service roles, use of 
performance measures and managing data 4.4 (N-76) 
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Academic health science library service roles 
Institutional settings for planning, evaluation, 



3.9 



<N«75) 



and data management 



3.7 
3.7 



(N-78) 
(N-78) 



Developing performance measures 



Managing per f ormance measure and strategic 



planning data 



3.9 
34 



(N-76) 
(N-76) 



Demonstration of the TMC MIS 



These means indicate that the topics were generally useful to participants and suggest that the 
objectives of the workshop were met 

Respondents were also asked to suggest educational follow-ups to this workshop. It was 
suggested mat opportunities (workshops or other activities) to provfde additional information on the 
following topics be provided: 

• Outcome measures 

• Methods for assessing value of services 

• How to collect data for new database 

• Methods for using a strategic plan for institutional positioning 

• How to develop performance measures and how to relate them to service roles 

• Service role development 

• The relationship of new measures to traditionally collected measures 

• MIS development 

Additional suggestions were: 

• The presentation of the same type of program at MLA 

• Similar regional workshops so that other staff could attend 

• A workshop where participants could bring actual documents for consultation 

• Provision of copies of the TMC MIS 

. • Once the manual is developed, a workshop on how to implement and use it 

• A workshop on field survey methods 

• A similar program for midlevel managers 

• Ongoing status reports on the project (including perhaps another workshop next year) 

• A summary of the workshop findings to be distributed to all participants 

These suggestions indicate that the workshop stimulated creative thinking and they provide some 
useful directions for AAHSLDs ongoing educational activities. 

Comments on the evaluation form reinforced the perception of the workshop leaders that the 
objectives were being met As the workshop leaders observed and interacted with participants 
throughout the day, it was dear that serious consideration of the topics in relationship to each 
participant's home institution was occurring. The processes discussed during the workshop are complex 
ones, and throughout the workshop it was dear that this complexity was recognized by participants 
and that they actively sought to make sense of than in terms of their own institution. 

WORKSHOP EXERCISES 

There were three group exercises during the course of the workshop. Due to the large size of the 
group and a desire to allow for interaction between participants who may not have known each other, 
the workshop leaders assigned participants to 12 groups. These groups worked together throughout the 
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day. The three exercises wh'xh built on topic* presented by the workshop leaders were the critique ind 
development of vision statements, the assessment of institutional settings, and die development of 
performance measures. 

VTSION STATEMENT CRITIQUE AND DEVELOPMENT 

Prior to the workshop, the study team had. In consultation with several academic health 
science librarians, developed a set of draft vision statements. A vision statement is a description of a 
possible future state or set of functions for an academic health sdence library. A primary purpose of the 
development of a vision statement is to define and describe a vision of what the library might be in the 
future. Such visions are useful for strategic planning, since they enable a library to identify which 
visions are feasible for the library as weQ as moat beneficial for the library and its clientele and then 
allow the library to take appropriate steps to move towards the vision. 

The goal of the workshop exerdse (attachment 2) was to increase participant awareness of the 
role of vision statements in strategic planning and allow participants to practice creating a new vision 
statement or refining an existing one. Workshop participants were asked to briefly evaluate the draft 
set of visions and then work at modifying one or creating a new one. 

General comments on the vision statements as a whole suggested that they were not visionary 
enough; many participants felt their library was already doing many of the activities included in the 
Example Library Activities sections. A number of groups also suggested that the visions needed to 
emphasize users, education, and research more than they did. A number of additional visions were also 
suggested. They were: state-wide or regional resource, archival center, quality filtering, research 
support and research initiator, and knowledge and information navigator. 

Additional environmental and library assumptions were generated by the groups for particular 
vision statements. These assumptions provided valuable Information to the study team about the 
current constraints for academic health science libraries which need to be considered in planning 
activities. The additional assumptions also pointed out quite dearly that each library faces Its own 
set of institutional constraints and that vision statement development and implementation can only be 
successful when a library's specific institutional situation is understood. 

Significant editorial comments on the statements were made as well. These comments are not 
reported here but should be taken into consideration in revisions of the statements. An important 
question relating to vision statement design was made by one group which wondered whether the 
statement should include just possible goals and activities or present highly specific and precise goals. 
This group also questioned the relationship between vision and mission statements. How closely 
related should they be? These questions suggest that the study team needs to focus future educational 
activities on vision and mission statement development activities. 

Groups worked during the exerdse on revising a vision statement or developing a new one. Five 
groups worked on revising the vision statement entitled "Information Resource Manager." A suggested 
name change was to Information Resources Leadership to suggest that the library played an active role 
in designing the'information architecture" as in developing institutional information policy. 
Additional activities which should be added included library participation in informing the 
community about resources, die management of electronic resources, and the library itself as an 
recognized institutional information resource. 

Three groups worked on revision of die vision statement entitled "Educational and Curriculum 
Support Center." The comment was made that the vision as it now stands presents the library as 
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technical support rather than as in intellecti ! partner in educational activities. It was suggested 
that a more visionary role for the library would be, as one group pit it, "as provider of leadership in 
the development, design, and approach to information literacy and knowledge management skills for 
lifelong learning and as a partner in problem-based learning initiatives." Additional activities 
included library staff involved in leaching activities and in the development of faculty teaching skills 
particularly in the area of knowledge management skills. 

Two groups revised the vision statement entitled "Electronic Networked Library." An 
additional enhancement to the vision presented was to include the library's role as creator and 
developer of electronic information. 

One group revised the statement entitled Information Technology Innovator." To make the 
statement more visionary, the group felt it needed to capture the library's involvement in the 
development of Information technology not just the d e monst ration of existing products. An important 
environmental as s ump tion which was missing from the existing list was the ever increasing proportion 
of the library budget that automation activities consume. 

One group revised the vision entitled "Information Support Center." The group felt the 
statement needed to be fleshed out presenting a more proactive picture of the library's involvement in 
institutional activities such as health care and information technology. 

One group developed a new vision statement which they entitled "Knowledge Navigator. The 
vision focusaed on the changing mode of distribution of information which increased networking 
capabilities. In this vision the library goals are to play an active role in helping users select from a 
broad array of resources, to collaborate with others in institution to provide gateways to information, to 
work closely with users to determine their needs, and to make decisions about budget and services based 
on these needs. Library assumptions induded the existence of network connections, staff skills and 
ability to develop user friendly front ends to systems, to determine user needs, as well as to perform 
necessary training and promotional activities. Collection resources would also be reallocated to support 
networked information and infrastructure. 

This exercise indicated to the study team the importance of the process of developing vision 
statements. It may not be possible to develop a set of possible visions which are applicable to all 
AAHSLD member libraries and which present visions which are "visionary enough." Workshop 
participants made it clear that institutional settings play a large role in what visions are truly visions 
for the library and not Just refinements of existing service roles, and also in determining what visions 
are attainable or applicable in the institution. It is feasible however, to develop a model of die vision 
development process making it possible for an individual library to create and use their own vision 
statements in the library's planning activities. 

INSTITUTIONAL SETTING EXERCISE 

During the various focus groups held in conjunction with the project, it became dear to the study 
team that an understanding of the institutional environment in which an academic health science 
library resided was critical to an understanding of how strategic planning would be performed in that 
library. A number of factors were discussed by focus group participants and served as the background for 
the workshop institutional setting session and exercise. 

The goal of the exercise was to create an awareness for each participant of the institutional 
factors which influenced his or her ability to perform planning functions. A second goal was to develop 
an understanding of which factors might have the largest impact on that planning effort In the 
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exercise, participants first developed a brief description of their institutional setting after addressing 
a series of questions about the institution (attachment 3). The group exercise asked each group to discuss 
the similarities and differences be t ween institutional settings and consider which were most important 
to planning efforts (attachment 4). 

86 assessments of individual institutional settings were collected by the study team. Given time 
constraints, these have not been analyzed, but the study team believes that they will provide 
important information about the types of environments in which academic health science libraries 
reside and contribute to efforts to develop a typology of institutional settings which will be 
incorporated into the planning manual. 

The group exerdse did provide insight into some of the institutional factors participants 
considered important for planning. Factors mentioned were: 

• The reporting brie tor the library 

• The institutional culture 

• Whether the institution was public or private 

• Whether decision making was centralized or decentralized 

• The background of top decision makers 

• The level of available funding 

• The source of funding (e.g., state, research, clinical) 

• The degree of library independence from its administration 

• The clientele served 

• The strategic iseues and priority of missions within the institution 

• The age of the institution 

• The commitment to planning 

• Whether the institution has single or multiple programs 

• The relationship between the hospital and the medical school 

• Whether the institution had 2 or 4 year programs 

This list extends that developed by the study team and provides additional factors which must be 
considered in institutional assessment and its associated impact on planning activities. 

Participants also tried to rank these factors, with some difficulty. Three groups said that the 
Institutional settings in the group were too diverse to come to a consensus. Other groups did develop 
rankings but generally there was not much overlap between the rankings. One factor, reporting lines did 
appear as one of the top three factors for 7 groups. Other factors also appeared on several ranking lists, 
resource availability (5), clientele served (3), whether the institution was public or private (3), and 
institutional priorities (3). 

Croups also tried to develop several typical institutional settings that encompassed the 
individual settings of group members. Some groups took a matrix approach and listed lh? factors which 
were important and then suggested that there was an institutional setting for each cell of the matrix. 
Others listed brief descriptions of the settings themselves. Since these varied results are hard to 
synthesize and summarize, the arts we. of each jroup which explored the question is listed below. 



Factors: Community based medical school as compared with 2 and 4 year programs 
Single vs. multiple programs 
Report to Dean or report to University library system 
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Factors: Public vs. private institution 
Age of institution 

Factors: Reporting line 

Public vs. private 



Settings: Reports to Medical Center, funded by Health Science Center, very little library 
information required 

Large research library reporting to University library; 
Dual reporting lines 

Settings: Part of University library system 
Private medical school 

Separate medical school campus of a University library 

Settings: Report to Dean 

Report to University library 
Mix of the first two 

Settings: Public, multi-program schools 

Private schools reporting to the Dean 
Reporting to the University Library 

Settings: Free standing medical institution 

Medical institution as part of a larger academic institution 

Libraries which support clinical activities in addition to research and education 

The final part of the exercise had groups suggest strategies which might have a favorable 
impact on the institutional environment in which the libraries operated. These suggestions fell into 
several categories: 

• P.omotional strategies 

develop a list of benefits and publicize them 

* Demonstration strategies 

demonstrate die cost effectiveness of planning 
demonstrate that planning can produce additional revenue 
demonstrate the library's effectiveness as planner 
demonstrate that planning can further the goals of the institution 
demo ns tr a te that the library is a leader in institutional planning 
demonstrate that the library contributes to HSC accreditation 



• Involvement strategies 

be involved in the institutional decision making process 
get known formally and informally 

start an in-house educational program and involve library staff in planning activities 

involve library users in planning activities 

target influential people and engage them in the process 

communicate 



• 'institutional awareness strategies 



gain aooEM to curricular and research institutional information 
learn to use aid work within the institutional culture 
understand economic realities and projections 

• Other strategic* 

demythologize electronic media 
use directed/project oriented/action planning 
run pilot projects to see if goals can be achieved 
use external consultants 

identify successful planning efforts from other similar institutions and use as models 

Several of these strat e gi e s were mentioned more than once. Many of them point to the need for 
increasing awareness of what drives the institution and also towards the need to develop measures 
which can be used to demonstrate the value of the library and its services. 

PERFORMANCE MEASURE DEVELOPMENT EXERCISE 

A third component of planning for academic health science libraries is that of measurement and 
the collection and analysis of data which can be used to evaluate the value of library services and 
indicate whether library goals have been met. The study team s experience, reinforced by findings from 
the focus groups, suggested that the development of performance measures linked to vision statements is 
one of the most appropriate ways academic health science libraries can demonstrate their value to 
their Institutions. Therefore, the third exercise of the workshop focussed on developing performance 
measures. 

The goal of the exercise was to give participant a sense of the difference between output and 
outcome measures, as well as to create an awareness of the proceM of linking measures to goals suggested 
by vision statements. In the exercise, participants developed measures for the vision statement they 
had analyzed earlier in the day, and considered sources of data for those measures (attachment 5). One 
of the major difficulties encountered by the groups was establishing the difference between an output 
measure and an outcome measure. By the end of the exercise, it appeared to workshop leaders that 
many of the group. Had come to understand the difference and had begun to develop performance 
measures in both categories. 

Listed below are the outcomes and/or measures reported by participants which might 
determine whether a vision had been attained. The study team has not yet evaluated the utility of 
measures listed betow . Some groups focussed on developing outcomes while others suggested actual 
measures. Where it was possible, measures are listed beneath the outcome they might indicate. The*; 
suggested measures are a useful step toward* the development of a more com p rehensive set of measures 
by the study team. 

Information Resource Manager 

Outcome: Library is seen as a player in information resource management 
Library is key component in organizational structure 

Number of times library is consulted on info nation resource management issues 
Number of committees, etc on which library staff participate 
Percentage of participation on these committees 

Number of initiatives undertakes (eg. current awareness, quality filtering) 
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Outcome: Satisfaction with Service 
Documents delivered 
Rate of complaints 

Percentage of administrative queries answered 

Questions asked and answered (with a connection to the collections and the number of 

Accreditation reports 
Outcome: Timeliness of Service 

Turnaround time 

Outcome: Educational force in institution 

Behavioral change as a result on information provided 
Re s ponse s to evaluation of educational activities 
Number of participants in educational activities 
Orientation and educational program statistics 
Computer searching statistics 

Outcome, library collection meets current and future needs 

Holdings in specific areas 

Number of subscriptions 

Amount of money spent on collections 

Outcome: Library has promoted or facilitated access to information 

ILL statistics 

Collection use statistic , 

Catalog use statistics 

Computer search statistics 

Number of clients connected to network 

Percentage of users with access to a workstation 

Gate count compared to network access to information 

Percentage of possible users using services 

Cost to circulate an item vs. cost to access network 

Outcome: Library is perceived as a change agent 

Existence of planning document mat includes library as major player 

Outcome: Campus information systems are connected 

Number of types of academic health science information available at a single 
workstation 

Outcome Increased economy for institution 

Amount of redundancy of resources at institution 
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Knowledge Navigator 



Possible measures: 

Number of times the library is asked to teach about available information in courses 
Number of end users to totai user population 
Number of gateway transactions 
Number of repeat users of services 
Number of documents requ es t ed 

Number of people who ask library to serve as quality filter 

Education and Curriculum Support Center 

Outcome library acknowledged as leader in curricular activities 

Proportion of times library services are included in grant proposals 
Amount of library impact on decision making processe s related to curriculum 
Collaboration of library with faculty on instructional modules 
Number of required and elective courses developed by library 
Number of collaborative programs 

Residency first choice* as a result of library role in curriculum 
The quality of student performance due to library services 
Number of students involved in library developed courses 
Percentage of institutional budget for library curricular support services 

Outcome: Library has required resources for curricular development 

Faculty satisfaction level with services 

Percentage of faculty requests on hand at time of faculty request 

Amount of use of instructional materials 

Amount of use of curricular support center 

Outcome: Faculty change instructional behavior and incorporate new teaching techniques 

Number of courses with an information component 

Outcome: Teaching effectiveness is considered in promotion decisions 

Outcome: Students are better educated 

Satisfaction of students in courses utilizing new teaching methods 
Success of students as life-long learners 
Residency placement 

Electronic Networked Library 

Outcome: Systems available on the network are viewed as primary information resources 

Percentage and number of users on network 

Amount and frequency of use of different systems on the network 

Degree of success of interaction in satisfying an information need 
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Number of new users of systems 
Number of unlinked users 

Amount of influence on patient care decision making 
User satisfaction with services 
Types of q u estio ns from users 

Along with outcomes and measures, groups were asked to consider sources of data and data 
collection methods. Groups generated the following sources and/or techniques: 

Ethnographic research 
Critical incident methods 
Cost-benefit analysis 

Measurement of multiple year trends (especially expenditures) 

User self-reports or testimonials 

AAHSLD Annual Statistics 

Longitudinal study of students 

Information c ompeten cy exams 

Use of automated transaction logs 

Before and after measures 

Annual and departmental reports 

The abundance of proposed measures and suggested techniques for data collection indicates that 
AAHSLD member libraries are interested in and aware of the need to demonstrate the value of the 
library to the institution in new ways. In fact, many are, collecting, or are ready to begin collecting 
data which could be used to develop performance measures. 

MANAGEMENT INFORMATION SYSTEMS 

The final session of the workshop was a demonstration of a library management information 
system. Damon Camille demonstrated the MIS that the Texas Medical Center has been developing for 
the last two years. He highlighted some of the functions and indicated sources of data including online 
spreadsheets, direct input of transactional data (e.g., ILL requests), and migration of data from the 
library's other automated systems. 

A number of issues were raised by workshop participants during this demonstration. These 

were: 

• What level of cost and time can a library devote to MIS development? 

• Does the library have the technical staff necessary to develop an MIS? 

• What types of data should be in the MIS? How can one assure the reliability of that data? 

• What types of data should be in the AAHSLD database as opposed to the local MIS? 

• Are library staff willing to use the system to record information and to use it to answer 
management questions? 

• b it feasible to take an existing MIS and import it into the local environment? What are the 
compatibility concerns? 
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These questions indicate a number or concerns that need to be addressed as the study team explores the 
utilization of MIS in the planning process. The study team will also need to move AAHSLD librarians 
beyond these questions to consider the linkages between the MIS and planning activities such as the 
development and utilization of performance measures, the assessment of institutional settings, and the 
measurement of progress towards library visions. 

RECOMMENDATIONS FOR FUTURE ACTION 

The workshop results suggest a number of areas for future action on the part of the study team. 
These are: 

• Continuing exploration of the linkages between institutional settings, vision statements, 
performance measures, and management information systems (MIS) in order to facilitate 
planning in AAHLSD member libraries. 

• Refinement of the dimensions of institutional settings which affect planning activities and 
the development of typologies of institutional settings. 

• Better explication of the components that describe vision statements and the development of 
a model of the vision statement creation and utilization process. 

• Refinement and testing of some of the performance measures suggested by participants. 

• Consideration of the degree to which a Txasic" turnkey MIS software program could be 
utilized by all or most AAHSLD libraries. 

• Exploration of what data elements should be in the AAHSLD database as opposed to the 
MIS in individual libraries. 

• The ongoing education of AAHSLD member s in planning activities. 

These suggestions for future action will be integrated into the tasking for the remainder of the project. 

One of the areas in which further action is needed is in the exploration of linkages between 
vision statement development, institutional settings, and performance measures. Workshop 
participants were quite knowledgeable about several of the components of planning, but less clear on 
how all the pieces might fit together to create a strategic posture for the library within the institution. 
Given the current uncertain environment for academic health science libraries, it is critical that all 
planning elements work together in a synergistic fashion to improve the library's visibility and 
performance in the institution. The study team should concentrate efforts on investigating the nature of 
the linkages between missions, visions, institutional settings, and measurement in order to develop a 
planning manual that can truly facilitate planning in academic health science libraries. 

Another area for future action is in refinement of the study team's understanding of the critical 
dimensions of institutional settings. Workshop participants suggested several key dimensions but these 
were not unanimously recognized by participants as being important, and while some groups did define 2- 
3 typical institutional settings, these settings were again not uniform across the groups. It seems clear 
that the study team needs to continue to work to tease out the dimensions most relevant to library 
planning activities in order to develop a typology of settings. One possible source of data is the 
individual institutional setting statements developed by workshop participants. Once the relevant 
dimensions are determined, they need to be combined to create a typology of institutional types. A 
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library could umm which institutional setting type fit best and then use the corresponding planning 
techniques, visions, and measures which applied . 

As participants worked through the vision statement development exercise, it became dear 
that the draft statements needed to be more visionary, and in feet may have represented some existing 
academic health science libraries service roles. In addition, participants suggested that many of the 
environmental assumptions and library assumptions might be specific to a particular institutional 
setting. These observations as well as the number of editorial comments made about the format, 
structure, and level of detail of the statement led the study team to the conclusion that future efforts 
needed to move away from actual vision statement devebpment towards the development of a process 
and model for creation and utilization of such visions. Additionally, the study team needs to determine 
the necessary components of vision statements; those components that make them understandable and 
useful to a library and its administration. 

Another area for action is the validation of suggested performance measures and the 
development of additional ones. These measures need to be developed in light of action taken by the 
study team to develop a typology of institutional settings and possible visions since the measures serve 
as indicators of library value in relationship to its setting and the vision to be attained. 

The study team also needs to explore the feasibility of the development of a "generic" MIS for 
AAHSLD member libraries. Is it possible to determine the types of data and reports which would be 
used by the majority of libraries for management purposes? Any efforts to develop an MIS need to take 
into account the AAHSLD database and the role it serves for the membership. 

Finally, it is dear that additional educational opportunities need to be available for 
AAHSLD member s . Comments on the workshop evaluation form suggest several areas for further 
activities as well as possible venues. It was apparent that some participants were less convinced then 
others about the importance of and need for strategic planning. As the study team continues its work 
exploring the areas indicated above, as well as needs addressed in other reports, it is clear that the 
membership needs to be apprised of this work both to build support for the project and its objectives and 
also to provide relevant information to improve planning activities in the libraries. Educational 
settings also provide valuable opportunities for the exchange of information, enabling the membership 
to help guide the project and provide their feedback to the study team. 
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